2024-04-22T06:56:37.676 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-22T06:56:37.676 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/snaps}} --verbose --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157 --name leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi -- /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157/orig.config.yaml 2024-04-22T06:56:37.726 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T06:56:37.818 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157 branch: wip-lusov-quiescer description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/snaps}} email: null first_in_suite: false job_id: '7668157' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 4362 sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 sleep_before_teardown: 0 subset: 222/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJW4DH9/OuIrQPDDxiqj26fgsnmpfYkm+x/wb5NNR8Y5p1CnQ3FLcnYlHCQjc5E30uENfANhCb/KTxaQcwDa54w= smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdaKEfSykVzVIZHdC406/XHcSmPSaAcpxjPhJfeCQCrhbd2qVl9nSXdPpBrNIGCrpVzfbG/iCeBBGWamxCjO4k= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - quiescer: null - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-22_05:52:53 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-22T06:56:37.819 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa; will attempt to use it 2024-04-22T06:56:37.819 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks 2024-04-22T06:56:37.819 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-22T06:56:37.821 INFO:teuthology.task.internal:Checking packages... 2024-04-22T06:56:37.843 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '86f7587a5a09af35f0895e1a2d08527638fae697' 2024-04-22T06:56:37.843 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T06:56:37.843 INFO:teuthology.packaging:ref: None 2024-04-22T06:56:37.843 INFO:teuthology.packaging:tag: None 2024-04-22T06:56:37.843 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-22T06:56:37.843 INFO:teuthology.packaging:sha1: 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T06:56:37.843 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-22T06:56:38.009 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.g86f7587a 2024-04-22T06:56:38.010 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-22T06:56:38.016 INFO:teuthology.task.internal:no buildpackages task found 2024-04-22T06:56:38.016 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-22T06:56:38.021 INFO:teuthology.task.internal:Saving configuration 2024-04-22T06:56:38.038 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-22T06:56:38.044 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T06:56:38.076 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:46.230443', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJW4DH9/OuIrQPDDxiqj26fgsnmpfYkm+x/wb5NNR8Y5p1CnQ3FLcnYlHCQjc5E30uENfANhCb/KTxaQcwDa54w='} 2024-04-22T06:56:38.106 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:46.231846', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdaKEfSykVzVIZHdC406/XHcSmPSaAcpxjPhJfeCQCrhbd2qVl9nSXdPpBrNIGCrpVzfbG/iCeBBGWamxCjO4k='} 2024-04-22T06:56:38.106 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-22T06:56:38.112 INFO:teuthology.task.internal:roles: ubuntu@smithi044.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-22T06:56:38.112 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-22T06:56:38.112 INFO:teuthology.run_tasks:Running task console_log... 2024-04-22T06:56:38.199 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe249de80d0>, signals=[15]) 2024-04-22T06:56:38.199 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-22T06:56:38.204 INFO:teuthology.task.internal:Opening connections... 2024-04-22T06:56:38.205 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T06:56:38.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:56:38.295 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T06:56:38.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T06:56:38.394 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-22T06:56:38.401 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-04-22T06:56:38.420 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-04-22T06:56:38.420 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-04-22T06:56:38.477 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-04-22T06:56:38.477 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="9" 2024-04-22T06:56:38.477 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="9" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el9" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:LOGO="fedora-logo-icon" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T06:56:38.478 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T06:56:38.479 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-04-22T06:56:38.515 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-04-22T06:56:38.533 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-04-22T06:56:38.533 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-04-22T06:56:38.589 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T06:56:38.590 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T06:56:38.591 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-04-22T06:56:38.623 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-22T06:56:38.632 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-22T06:56:38.708 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-22T06:56:38.708 DEBUG:teuthology.orchestra.run.smithi044:> test '!' -e /home/ubuntu/cephtest 2024-04-22T06:56:38.710 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-04-22T06:56:38.726 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-22T06:56:38.732 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-22T06:56:38.732 DEBUG:teuthology.orchestra.run.smithi044:> test -z $(ls -A /var/lib/ceph) 2024-04-22T06:56:38.767 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-04-22T06:56:38.797 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-22T06:56:38.865 INFO:teuthology.run_tasks:Running task kernel... 2024-04-22T06:56:38.879 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-22T06:56:38.879 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-22T06:56:38.879 DEBUG:teuthology.orchestra.run.smithi044:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T06:56:38.880 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T06:56:38.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:56:38.897 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-04-22T06:56:38.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:56:38.898 DEBUG:teuthology.orchestra.run.smithi044:> uname -r 2024-04-22T06:56:38.954 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-437.el9.x86_64 2024-04-22T06:56:38.954 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.14.0-437.el9.x86_64 2024-04-22T06:56:38.954 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-04-22T06:56:38.955 INFO:teuthology.orchestra.run.smithi044.stdout:5.14.0-437.el9.x86_64 2024-04-22T06:56:38.955 INFO:teuthology.task.kernel:Running kernel on smithi044: 5.14.0-437.el9.x86_64 2024-04-22T06:56:38.955 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum install -y kernel 2024-04-22T06:56:42.009 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-04-22T06:56:42.152 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-04-22T06:56:45.544 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-04-22T06:56:45.810 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 7.1 MB/s | 19 MB 00:02 2024-04-22T06:56:49.611 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-22T06:56:49.994 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-22T06:56:51.153 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-04-22T06:56:51.461 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-22T06:56:53.104 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-22T06:56:58.346 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-22T06:57:00.277 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T06:57:00.277 INFO:teuthology.orchestra.run.smithi044.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T06:57:00.365 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T06:57:00.366 INFO:teuthology.orchestra.run.smithi044.stdout:Nothing to do. 2024-04-22T06:57:00.366 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T06:57:00.483 DEBUG:teuthology.orchestra.run.smithi044:> echo no | sudo yum reinstall kernel || true 2024-04-22T06:57:01.047 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 06:56:58 AM UTC. 2024-04-22T06:57:01.178 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T06:57:01.189 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalling: 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 255 k 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 0 2024-04-22T06:57:01.190 INFO:teuthology.orchestra.run.smithi044.stderr:Operation aborted. 2024-04-22T06:57:01.259 INFO:teuthology.orchestra.run.smithi044.stdout:Is this ok [y/N]: 2024-04-22T06:57:01.259 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum reinstall -y kernel || true 2024-04-22T06:57:01.777 INFO:teuthology.orchestra.run.smithi044.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 06:56:58 AM UTC. 2024-04-22T06:57:01.898 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T06:57:01.898 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:57:01.899 INFO:teuthology.orchestra.run.smithi044.stdout: Package Architecture Version Repository Size 2024-04-22T06:57:01.899 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:57:01.899 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalling: 2024-04-22T06:57:01.899 INFO:teuthology.orchestra.run.smithi044.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:57:01.899 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:57:01.899 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T06:57:01.899 INFO:teuthology.orchestra.run.smithi044.stdout:================================================================================ 2024-04-22T06:57:01.899 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:57:01.899 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 255 k 2024-04-22T06:57:01.899 INFO:teuthology.orchestra.run.smithi044.stdout:Installed size: 0 2024-04-22T06:57:01.900 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T06:57:02.298 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-437.el9.x86_64.rpm 948 kB/s | 255 kB 00:00 2024-04-22T06:57:02.298 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T06:57:02.298 INFO:teuthology.orchestra.run.smithi044.stdout:Total 639 kB/s | 255 kB 00:00 2024-04-22T06:57:02.316 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T06:57:02.488 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T06:57:02.488 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T06:57:02.493 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T06:57:02.493 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T06:57:02.656 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T06:57:02.757 INFO:teuthology.orchestra.run.smithi044.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:57:02.758 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:03.508 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:03.508 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:57:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:57:04.246 INFO:teuthology.orchestra.run.smithi044.stdout:Reinstalled: 2024-04-22T06:57:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T06:57:04.246 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T06:57:04.246 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T06:57:04.440 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T06:57:04.488 INFO:teuthology.orchestra.run.smithi044.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T06:57:04.488 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T06:57:04.488 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T06:57:04.488 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-22T06:57:04.488 DEBUG:teuthology.orchestra.run.smithi044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T06:57:04.522 INFO:teuthology.orchestra.run.smithi044.stdout:ttyS1 2024-04-22T06:57:04.546 DEBUG:teuthology.parallel:result is None 2024-04-22T06:57:10.062 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-04-22T06:57:15.000 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-22T06:57:16.823 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T06:57:16.823 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T06:57:16.914 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-22T06:57:16.915 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-04-22T06:57:16.915 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-22T06:57:17.031 DEBUG:teuthology.orchestra.run.smithi070:> echo no | sudo yum reinstall kernel || true 2024-04-22T06:57:17.525 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 06:57:14 AM UTC. 2024-04-22T06:57:17.645 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-22T06:57:17.653 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:57:17.653 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-22T06:57:17.653 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:57:17.653 INFO:teuthology.orchestra.run.smithi070.stdout:Reinstalling: 2024-04-22T06:57:17.653 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:57:17.654 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:57:17.654 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-22T06:57:17.654 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:57:17.654 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:57:17.654 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 255 k 2024-04-22T06:57:17.654 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 0 2024-04-22T06:57:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:Operation aborted. 2024-04-22T06:57:17.717 INFO:teuthology.orchestra.run.smithi070.stdout:Is this ok [y/N]: 2024-04-22T06:57:17.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum reinstall -y kernel || true 2024-04-22T06:57:18.206 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:04 ago on Mon 22 Apr 2024 06:57:14 AM UTC. 2024-04-22T06:57:18.321 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-22T06:57:18.322 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:57:18.322 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-04-22T06:57:18.322 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:57:18.322 INFO:teuthology.orchestra.run.smithi070.stdout:Reinstalling: 2024-04-22T06:57:18.322 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T06:57:18.322 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:57:18.322 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-22T06:57:18.322 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-04-22T06:57:18.322 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:57:18.323 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 255 k 2024-04-22T06:57:18.323 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 0 2024-04-22T06:57:18.323 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-22T06:57:18.819 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-22T06:57:18.820 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-22T06:57:18.820 INFO:teuthology.orchestra.run.smithi070.stdout:Total 512 kB/s | 255 kB 00:00 2024-04-22T06:57:18.837 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-22T06:57:19.031 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-22T06:57:19.031 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-22T06:57:19.036 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-22T06:57:19.036 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-22T06:57:19.195 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-22T06:57:19.261 INFO:teuthology.orchestra.run.smithi070.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:57:19.262 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:19.978 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:19.978 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T06:57:20.678 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T06:57:20.679 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:57:20.679 INFO:teuthology.orchestra.run.smithi070.stdout:Reinstalled: 2024-04-22T06:57:20.679 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T06:57:20.679 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T06:57:20.679 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-22T06:57:20.841 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T06:57:20.887 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T06:57:20.888 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T06:57:20.888 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T06:57:20.888 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-22T06:57:20.888 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T06:57:20.922 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-04-22T06:57:20.940 DEBUG:teuthology.parallel:result is None 2024-04-22T06:57:20.940 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-22T06:57:20.948 INFO:teuthology.task.internal:Creating test directory... 2024-04-22T06:57:20.948 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T06:57:20.951 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T06:57:20.994 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-22T06:57:21.001 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-22T06:57:21.007 INFO:teuthology.task.internal:Creating archive directory... 2024-04-22T06:57:21.007 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T06:57:21.009 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T06:57:21.061 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-22T06:57:21.067 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-22T06:57:21.067 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T06:57:21.080 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T06:57:21.110 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:21.121 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:21.149 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:21.158 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T06:57:21.160 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-22T06:57:21.167 INFO:teuthology.task.internal:Configuring sudo... 2024-04-22T06:57:21.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T06:57:21.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T06:57:21.194 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-22T06:57:21.203 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-22T06:57:21.203 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:57:21.237 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T06:57:21.251 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:21.337 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:21.377 DEBUG:teuthology.orchestra.run.smithi044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:21.471 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:21.502 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T06:57:21.502 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T06:57:21.567 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:21.613 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T06:57:21.655 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:21.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T06:57:21.775 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T06:57:21.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T06:57:21.838 DEBUG:teuthology.orchestra.run.smithi044:> sudo service rsyslog restart 2024-04-22T06:57:21.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-04-22T06:57:21.878 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:57:21.918 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T06:57:22.357 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-22T06:57:22.364 INFO:teuthology.task.internal:Starting timer... 2024-04-22T06:57:22.364 INFO:teuthology.run_tasks:Running task pcp... 2024-04-22T06:57:22.375 INFO:teuthology.run_tasks:Running task selinux... 2024-04-22T06:57:22.383 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-22T06:57:22.383 DEBUG:teuthology.orchestra.run.smithi044:> sudo service auditd rotate 2024-04-22T06:57:22.454 INFO:teuthology.orchestra.run.smithi044.stdout:Rotating logs: 2024-04-22T06:57:22.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-04-22T06:57:22.508 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-04-22T06:57:22.510 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-22T06:57:22.510 DEBUG:teuthology.orchestra.run.smithi044:> /usr/sbin/getenforce 2024-04-22T06:57:22.560 INFO:teuthology.orchestra.run.smithi044.stdout:Permissive 2024-04-22T06:57:22.560 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-04-22T06:57:22.582 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-04-22T06:57:22.582 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi044.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi070.front.sepia.ceph.com': 'permissive'} 2024-04-22T06:57:22.582 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:57:22.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:57:22.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T06:57:22.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T06:57:22.647 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-22T06:57:22.647 DEBUG:teuthology.orchestra.run.smithi044:> sudo /usr/sbin/setenforce permissive 2024-04-22T06:57:22.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-04-22T06:57:22.711 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-22T06:57:22.720 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-22T06:57:22.736 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-22T06:57:22.737 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi044.front.sepia.ceph.com,smithi070.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-22T07:00:52.651 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi044.front.sepia.ceph.com'), Remote(name='ubuntu@smithi070.front.sepia.ceph.com')] 2024-04-22T07:00:52.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-22T07:00:52.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T07:00:52.737 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-04-22T07:00:52.813 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-04-22T07:00:52.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-04-22T07:00:52.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T07:00:52.897 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-04-22T07:00:52.977 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-04-22T07:00:52.977 INFO:teuthology.run_tasks:Running task clock... 2024-04-22T07:00:52.987 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-22T07:00:52.987 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T07:00:52.988 DEBUG:teuthology.orchestra.run.smithi044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:00:52.990 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T07:00:52.991 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:00:53.016 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T07:00:53.030 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T07:00:53.044 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T07:00:53.054 INFO:teuthology.orchestra.run.smithi044.stderr:sudo: ntpd: command not found 2024-04-22T07:00:53.057 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T07:00:53.064 INFO:teuthology.orchestra.run.smithi044.stdout:506 Cannot talk to daemon 2024-04-22T07:00:53.078 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T07:00:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-04-22T07:00:53.089 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-04-22T07:00:53.092 INFO:teuthology.orchestra.run.smithi044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T07:00:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T07:00:53.115 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T07:00:53.146 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:00:53.151 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:00:53.151 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-22T07:00:53.151 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:53.151 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:53.151 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:53.151 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:53.166 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:00:53.170 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:00:53.171 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-04-22T07:00:53.171 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:53.171 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:53.171 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:53.171 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:00:53.171 INFO:teuthology.run_tasks:Running task install... 2024-04-22T07:00:53.179 DEBUG:teuthology.task.install:project ceph 2024-04-22T07:00:53.180 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}} 2024-04-22T07:00:53.180 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'} 2024-04-22T07:00:53.180 INFO:teuthology.task.install:Using flavor: default 2024-04-22T07:00:53.185 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-22T07:00:53.186 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-22T07:00:53.186 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'tag': None, 'wait_for_package': False} 2024-04-22T07:00:53.186 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:00:53.189 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'tag': None, 'wait_for_package': False} 2024-04-22T07:00:53.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:00:53.344 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/ 2024-04-22T07:00:53.344 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.g86f7587a 2024-04-22T07:00:53.427 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/ 2024-04-22T07:00:53.427 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.g86f7587a 2024-04-22T07:00:53.535 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T07:00:53.535 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:00:53.535 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T07:00:53.561 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T07:00:53.561 DEBUG:teuthology.orchestra.run.smithi044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86f7587a5a09af35f0895e1a2d08527638fae697/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T07:00:53.605 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/86f7587a5a09af35f0895e1a2d08527638fae697/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T07:00:53.605 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:00:53.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T07:00:53.627 DEBUG:teuthology.orchestra.run.smithi044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T07:00:53.630 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T07:00:53.630 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/86f7587a5a09af35f0895e1a2d08527638fae697/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T07:00:53.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T07:00:53.701 DEBUG:teuthology.orchestra.run.smithi044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T07:00:53.764 INFO:teuthology.orchestra.run.smithi044.stdout:check_obsoletes = 1 2024-04-22T07:00:53.766 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum clean all 2024-04-22T07:00:53.768 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T07:00:53.831 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-04-22T07:00:53.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-04-22T07:00:53.998 INFO:teuthology.orchestra.run.smithi044.stdout:48 files removed 2024-04-22T07:00:54.025 DEBUG:teuthology.orchestra.run.smithi044:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T07:00:54.054 INFO:teuthology.orchestra.run.smithi070.stdout:48 files removed 2024-04-22T07:00:54.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T07:00:54.714 INFO:teuthology.orchestra.run.smithi044.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-04-22T07:00:54.755 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-04-22T07:00:55.139 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-22T07:00:55.139 INFO:teuthology.orchestra.run.smithi044.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-22T07:00:55.548 INFO:teuthology.orchestra.run.smithi044.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-22T07:00:55.556 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-04-22T07:00:56.156 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-22T07:00:56.367 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-22T07:00:57.907 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-22T07:00:58.183 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-22T07:01:02.142 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-04-22T07:01:02.368 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-22T07:01:03.942 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-22T07:01:05.277 INFO:teuthology.orchestra.run.smithi044.stdout:CentOS Stream 9 - Extras packages 9.5 kB/s | 16 kB 00:01 2024-04-22T07:01:06.519 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-04-22T07:01:11.534 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-22T07:01:13.616 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:01:13.616 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:01:13.622 INFO:teuthology.orchestra.run.smithi070.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T07:01:13.623 INFO:teuthology.orchestra.run.smithi070.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T07:01:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T07:01:13.624 INFO:teuthology.orchestra.run.smithi070.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T07:01:13.625 INFO:teuthology.orchestra.run.smithi070.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T07:01:13.639 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T07:01:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T07:01:13.689 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-04-22T07:01:13.697 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-04-22T07:01:13.697 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-04-22T07:01:13.697 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-04-22T07:01:13.697 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-04-22T07:01:13.697 INFO:teuthology.orchestra.run.smithi070.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T07:01:13.697 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 6.6 k 2024-04-22T07:01:13.697 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:01:13.697 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 890 k 2024-04-22T07:01:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 145 k 2024-04-22T07:01:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 1.5 M 2024-04-22T07:01:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 142 k 2024-04-22T07:01:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 3.5 M 2024-04-22T07:01:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 7.4 M 2024-04-22T07:01:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 49 k 2024-04-22T07:01:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 11 M 2024-04-22T07:01:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 49 M 2024-04-22T07:01:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 273 k 2024-04-22T07:01:13.698 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 763 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 225 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-shell x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 27 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 18 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 32 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 747 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 127 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T07:01:13.699 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 162 k 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 320 k 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 298 k 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 100 k 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 85 k 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 171 k 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-04-22T07:01:13.700 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.4 M 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 21 M 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 34 k 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 2.4 M 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 249 k 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 4.7 M 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 17 M 2024-04-22T07:01:13.701 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 16 k 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 25 k 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 164 k 2024-04-22T07:01:13.702 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 500 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T07:01:13.703 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T07:01:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T07:01:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T07:01:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T07:01:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T07:01:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T07:01:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T07:01:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T07:01:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T07:01:13.704 INFO:teuthology.orchestra.run.smithi070.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T07:01:13.705 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T07:01:13.705 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T07:01:13.705 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T07:01:13.705 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 45 k 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 135 k 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T07:01:13.706 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T07:01:13.707 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T07:01:13.708 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T07:01:13.709 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T07:01:13.710 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:01:13.711 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-04-22T07:01:13.711 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-04-22T07:01:13.711 INFO:teuthology.orchestra.run.smithi070.stdout:Install 148 Packages 2024-04-22T07:01:13.711 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 2 Packages 2024-04-22T07:01:13.711 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:01:13.711 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 218 M 2024-04-22T07:01:13.711 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-04-22T07:01:28.343 INFO:teuthology.orchestra.run.smithi044.stdout:Extra Packages for Enterprise Linux 933 kB/s | 21 MB 00:22 2024-04-22T07:01:29.291 INFO:teuthology.orchestra.run.smithi070.stdout:(1/150): ceph-19.0.0-3156.g86f7587a.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-22T07:01:30.025 INFO:teuthology.orchestra.run.smithi070.stdout:(2/150): ceph-fuse-19.0.0-3156.g86f7587a.el9.x8 1.2 MB/s | 890 kB 00:00 2024-04-22T07:01:30.108 INFO:teuthology.orchestra.run.smithi070.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.7 MB/s | 145 kB 00:00 2024-04-22T07:01:31.083 INFO:teuthology.orchestra.run.smithi070.stdout:(4/150): ceph-mds-19.0.0-3156.g86f7587a.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-04-22T07:01:31.770 INFO:teuthology.orchestra.run.smithi070.stdout:(5/150): ceph-mgr-19.0.0-3156.g86f7587a.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-04-22T07:01:31.967 INFO:teuthology.orchestra.run.smithi070.stdout:(6/150): ceph-base-19.0.0-3156.g86f7587a.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-04-22T07:01:33.459 INFO:teuthology.orchestra.run.smithi070.stdout:(7/150): ceph-mon-19.0.0-3156.g86f7587a.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-04-22T07:01:33.484 INFO:teuthology.orchestra.run.smithi044.stdout:lab-extras 50 kB/s | 1.7 kB 00:00 2024-04-22T07:01:35.667 INFO:teuthology.orchestra.run.smithi070.stdout:(8/150): ceph-osd-19.0.0-3156.g86f7587a.el9.x86 4.5 MB/s | 17 MB 00:03 2024-04-22T07:01:35.767 INFO:teuthology.orchestra.run.smithi070.stdout:(9/150): ceph-selinux-19.0.0-3156.g86f7587a.el9 252 kB/s | 25 kB 00:00 2024-04-22T07:01:35.786 INFO:teuthology.orchestra.run.smithi044.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:01:35.786 INFO:teuthology.orchestra.run.smithi044.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T07:01:35.798 INFO:teuthology.orchestra.run.smithi044.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T07:01:35.798 INFO:teuthology.orchestra.run.smithi044.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T07:01:35.799 INFO:teuthology.orchestra.run.smithi044.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T07:01:35.800 INFO:teuthology.orchestra.run.smithi044.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T07:01:35.802 INFO:teuthology.orchestra.run.smithi044.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T07:01:35.804 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T07:01:35.805 INFO:teuthology.orchestra.run.smithi044.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T07:01:35.868 INFO:teuthology.orchestra.run.smithi044.stdout:Dependencies resolved. 2024-04-22T07:01:35.877 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-22T07:01:35.877 INFO:teuthology.orchestra.run.smithi044.stdout: Package Arch Version Repository Size 2024-04-22T07:01:35.877 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-22T07:01:35.877 INFO:teuthology.orchestra.run.smithi044.stdout:Installing: 2024-04-22T07:01:35.877 INFO:teuthology.orchestra.run.smithi044.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T07:01:35.877 INFO:teuthology.orchestra.run.smithi044.stdout: ceph x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 6.6 k 2024-04-22T07:01:35.877 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:01:35.877 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 890 k 2024-04-22T07:01:35.877 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 145 k 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 1.5 M 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 142 k 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 3.5 M 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 7.4 M 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 49 k 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 11 M 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 49 M 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 273 k 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 763 k 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 225 k 2024-04-22T07:01:35.878 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 27 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 18 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 32 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 747 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 127 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T07:01:35.879 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 162 k 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 320 k 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 298 k 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 100 k 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 85 k 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 171 k 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrading: 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: librados2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.4 M 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 3.1 M 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout:Installing dependencies: 2024-04-22T07:01:35.880 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 21 M 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 34 k 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 2.4 M 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 249 k 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 4.7 M 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 17 M 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.g86f7587a.el9 ceph-noarch 16 k 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 25 k 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T07:01:35.881 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 164 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 500 k 2024-04-22T07:01:35.882 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2 x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 5.4 M 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T07:01:35.883 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 45 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common x86_64 2:19.0.0-3156.g86f7587a.el9 ceph 135 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T07:01:35.884 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T07:01:35.885 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T07:01:35.886 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T07:01:35.887 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T07:01:35.888 INFO:teuthology.orchestra.run.smithi044.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout:Installing weak dependencies: 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction Summary 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout:======================================================================================= 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout:Install 148 Packages 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout:Upgrade 2 Packages 2024-04-22T07:01:35.889 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:01:35.890 INFO:teuthology.orchestra.run.smithi044.stdout:Total download size: 218 M 2024-04-22T07:01:35.890 INFO:teuthology.orchestra.run.smithi044.stdout:Downloading Packages: 2024-04-22T07:01:37.150 INFO:teuthology.orchestra.run.smithi070.stdout:(10/150): ceph-radosgw-19.0.0-3156.g86f7587a.el 2.9 MB/s | 11 MB 00:03 2024-04-22T07:01:37.309 INFO:teuthology.orchestra.run.smithi070.stdout:(11/150): cephfs-mirror-19.0.0-3156.g86f7587a.e 1.4 MB/s | 225 kB 00:00 2024-04-22T07:01:37.401 INFO:teuthology.orchestra.run.smithi070.stdout:(12/150): cephfs-shell-19.0.0-3156.g86f7587a.el 296 kB/s | 27 kB 00:00 2024-04-22T07:01:37.493 INFO:teuthology.orchestra.run.smithi070.stdout:(13/150): libcephfs-devel-19.0.0-3156.g86f7587a 346 kB/s | 32 kB 00:00 2024-04-22T07:01:37.735 INFO:teuthology.orchestra.run.smithi070.stdout:(14/150): libcephfs2-19.0.0-3156.g86f7587a.el9. 3.0 MB/s | 747 kB 00:00 2024-04-22T07:01:37.818 INFO:teuthology.orchestra.run.smithi070.stdout:(15/150): libcephsqlite-19.0.0-3156.g86f7587a.e 1.9 MB/s | 164 kB 00:00 2024-04-22T07:01:37.902 INFO:teuthology.orchestra.run.smithi070.stdout:(16/150): librados-devel-19.0.0-3156.g86f7587a. 1.5 MB/s | 127 kB 00:00 2024-04-22T07:01:38.102 INFO:teuthology.orchestra.run.smithi070.stdout:(17/150): libradosstriper1-19.0.0-3156.g86f7587 2.4 MB/s | 500 kB 00:00 2024-04-22T07:01:39.719 INFO:teuthology.orchestra.run.smithi070.stdout:(18/150): ceph-common-19.0.0-3156.g86f7587a.el9 1.9 MB/s | 21 MB 00:10 2024-04-22T07:01:39.854 INFO:teuthology.orchestra.run.smithi070.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.g86 336 kB/s | 45 kB 00:00 2024-04-22T07:01:39.928 INFO:teuthology.orchestra.run.smithi070.stdout:(20/150): librgw2-19.0.0-3156.g86f7587a.el9.x86 2.9 MB/s | 5.4 MB 00:01 2024-04-22T07:01:40.028 INFO:teuthology.orchestra.run.smithi070.stdout:(21/150): python3-cephfs-19.0.0-3156.g86f7587a. 1.6 MB/s | 162 kB 00:00 2024-04-22T07:01:40.053 INFO:teuthology.orchestra.run.smithi070.stdout:(22/150): python3-ceph-common-19.0.0-3156.g86f7 676 kB/s | 135 kB 00:00 2024-04-22T07:01:40.179 INFO:teuthology.orchestra.run.smithi070.stdout:(23/150): python3-rados-19.0.0-3156.g86f7587a.e 2.1 MB/s | 320 kB 00:00 2024-04-22T07:01:40.237 INFO:teuthology.orchestra.run.smithi070.stdout:(24/150): python3-rbd-19.0.0-3156.g86f7587a.el9 1.6 MB/s | 298 kB 00:00 2024-04-22T07:01:40.271 INFO:teuthology.orchestra.run.smithi070.stdout:(25/150): python3-rgw-19.0.0-3156.g86f7587a.el9 1.1 MB/s | 100 kB 00:00 2024-04-22T07:01:40.354 INFO:teuthology.orchestra.run.smithi070.stdout:(26/150): rbd-fuse-19.0.0-3156.g86f7587a.el9.x8 729 kB/s | 85 kB 00:00 2024-04-22T07:01:40.846 INFO:teuthology.orchestra.run.smithi070.stdout:(27/150): rbd-nbd-19.0.0-3156.g86f7587a.el9.x86 347 kB/s | 171 kB 00:00 2024-04-22T07:01:40.980 INFO:teuthology.orchestra.run.smithi070.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 256 kB/s | 34 kB 00:00 2024-04-22T07:01:41.172 INFO:teuthology.orchestra.run.smithi070.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3156.g86f7587 742 kB/s | 142 kB 00:00 2024-04-22T07:01:41.255 INFO:teuthology.orchestra.run.smithi070.stdout:(30/150): rbd-mirror-19.0.0-3156.g86f7587a.el9. 3.1 MB/s | 3.1 MB 00:00 2024-04-22T07:01:43.381 INFO:teuthology.orchestra.run.smithi070.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.g86f75 1.6 MB/s | 3.5 MB 00:02 2024-04-22T07:01:43.489 INFO:teuthology.orchestra.run.smithi070.stdout:(32/150): ceph-mgr-diskprediction-local-19.0.0- 3.3 MB/s | 7.4 MB 00:02 2024-04-22T07:01:43.564 INFO:teuthology.orchestra.run.smithi070.stdout:(33/150): ceph-mgr-rook-19.0.0-3156.g86f7587a.e 659 kB/s | 49 kB 00:00 2024-04-22T07:01:43.589 INFO:teuthology.orchestra.run.smithi070.stdout:(34/150): ceph-mgr-modules-core-19.0.0-3156.g86 1.2 MB/s | 249 kB 00:00 2024-04-22T07:01:43.631 INFO:teuthology.orchestra.run.smithi070.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3156.g8 247 kB/s | 16 kB 00:00 2024-04-22T07:01:43.766 INFO:teuthology.orchestra.run.smithi070.stdout:(36/150): ceph-volume-19.0.0-3156.g86f7587a.el9 1.5 MB/s | 273 kB 00:00 2024-04-22T07:01:43.849 INFO:teuthology.orchestra.run.smithi070.stdout:(37/150): cephadm-19.0.0-3156.g86f7587a.el9.noa 3.4 MB/s | 763 kB 00:00 2024-04-22T07:01:43.874 INFO:teuthology.orchestra.run.smithi070.stdout:(38/150): cephfs-top-19.0.0-3156.g86f7587a.el9. 168 kB/s | 18 kB 00:00 2024-04-22T07:01:44.016 INFO:teuthology.orchestra.run.smithi070.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 481 kB/s | 80 kB 00:00 2024-04-22T07:01:44.077 INFO:teuthology.orchestra.run.smithi044.stdout:(1/150): ceph-19.0.0-3156.g86f7587a.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-22T07:01:44.100 INFO:teuthology.orchestra.run.smithi070.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-22T07:01:44.159 INFO:teuthology.orchestra.run.smithi070.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-04-22T07:01:44.184 INFO:teuthology.orchestra.run.smithi070.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-22T07:01:44.234 INFO:teuthology.orchestra.run.smithi070.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-22T07:01:44.284 INFO:teuthology.orchestra.run.smithi070.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-22T07:01:44.335 INFO:teuthology.orchestra.run.smithi070.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-22T07:01:44.421 INFO:teuthology.orchestra.run.smithi070.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-22T07:01:44.467 INFO:teuthology.orchestra.run.smithi070.stdout:(47/150): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-04-22T07:01:44.586 INFO:teuthology.orchestra.run.smithi044.stdout:(2/150): ceph-fuse-19.0.0-3156.g86f7587a.el9.x8 1.7 MB/s | 890 kB 00:00 2024-04-22T07:01:44.604 INFO:teuthology.orchestra.run.smithi070.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-04-22T07:01:44.667 INFO:teuthology.orchestra.run.smithi070.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-22T07:01:44.667 INFO:teuthology.orchestra.run.smithi044.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.9 MB/s | 145 kB 00:00 2024-04-22T07:01:44.719 INFO:teuthology.orchestra.run.smithi070.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 370 kB/s | 21 kB 00:00 2024-04-22T07:01:44.836 INFO:teuthology.orchestra.run.smithi070.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-22T07:01:44.887 INFO:teuthology.orchestra.run.smithi070.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-04-22T07:01:44.954 INFO:teuthology.orchestra.run.smithi070.stdout:(53/150): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 943 kB 00:00 2024-04-22T07:01:45.004 INFO:teuthology.orchestra.run.smithi070.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-22T07:01:45.054 INFO:teuthology.orchestra.run.smithi070.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-04-22T07:01:45.138 INFO:teuthology.orchestra.run.smithi070.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-22T07:01:45.197 INFO:teuthology.orchestra.run.smithi070.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 776 kB/s | 45 kB 00:00 2024-04-22T07:01:45.370 INFO:teuthology.orchestra.run.smithi044.stdout:(4/150): ceph-mds-19.0.0-3156.g86f7587a.el9.x86 3.3 MB/s | 2.4 MB 00:00 2024-04-22T07:01:45.423 INFO:teuthology.orchestra.run.smithi070.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-04-22T07:01:45.530 INFO:teuthology.orchestra.run.smithi070.stdout:(59/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.6 MB/s | 3.0 MB 00:00 2024-04-22T07:01:45.580 INFO:teuthology.orchestra.run.smithi070.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 511 kB/s | 25 kB 00:00 2024-04-22T07:01:45.631 INFO:teuthology.orchestra.run.smithi070.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-22T07:01:45.656 INFO:teuthology.orchestra.run.smithi070.stdout:(62/150): libtool-2.4.6-45.el9.x86_64.rpm 2.4 MB/s | 582 kB 00:00 2024-04-22T07:01:45.715 INFO:teuthology.orchestra.run.smithi070.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-22T07:01:45.740 INFO:teuthology.orchestra.run.smithi070.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-22T07:01:45.757 INFO:teuthology.orchestra.run.smithi070.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 841 kB/s | 35 kB 00:00 2024-04-22T07:01:45.903 INFO:teuthology.orchestra.run.smithi044.stdout:(5/150): ceph-mgr-19.0.0-3156.g86f7587a.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-04-22T07:01:46.499 INFO:teuthology.orchestra.run.smithi070.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 5.6 MB/s | 4.1 MB 00:00 2024-04-22T07:01:46.766 INFO:teuthology.orchestra.run.smithi070.stdout:(67/150): openblas-openmp-0.3.21-2.el9.x86_64.r 4.6 MB/s | 4.7 MB 00:01 2024-04-22T07:01:46.824 INFO:teuthology.orchestra.run.smithi070.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-04-22T07:01:46.883 INFO:teuthology.orchestra.run.smithi070.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-22T07:01:47.008 INFO:teuthology.orchestra.run.smithi070.stdout:(70/150): postgresql-contrib-13.14-1.el9.x86_64 3.7 MB/s | 911 kB 00:00 2024-04-22T07:01:47.225 INFO:teuthology.orchestra.run.smithi070.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-04-22T07:01:47.228 INFO:teuthology.orchestra.run.smithi044.stdout:(6/150): ceph-mon-19.0.0-3156.g86f7587a.el9.x86 3.5 MB/s | 4.7 MB 00:01 2024-04-22T07:01:47.275 INFO:teuthology.orchestra.run.smithi070.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-22T07:01:47.701 INFO:teuthology.orchestra.run.smithi070.stdout:(73/150): postgresql-server-13.14-1.el9.x86_64. 7.1 MB/s | 5.8 MB 00:00 2024-04-22T07:01:47.759 INFO:teuthology.orchestra.run.smithi070.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-22T07:01:47.818 INFO:teuthology.orchestra.run.smithi070.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-22T07:01:47.876 INFO:teuthology.orchestra.run.smithi070.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-04-22T07:01:47.935 INFO:teuthology.orchestra.run.smithi070.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-04-22T07:01:47.994 INFO:teuthology.orchestra.run.smithi070.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-22T07:01:48.052 INFO:teuthology.orchestra.run.smithi070.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 596 kB/s | 35 kB 00:00 2024-04-22T07:01:48.162 INFO:teuthology.orchestra.run.smithi044.stdout:(7/150): ceph-base-19.0.0-3156.g86f7587a.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-04-22T07:01:48.211 INFO:teuthology.orchestra.run.smithi070.stdout:(80/150): python3-babel-2.9.1-2.el9.noarch.rpm 6.4 MB/s | 6.0 MB 00:00 2024-04-22T07:01:48.270 INFO:teuthology.orchestra.run.smithi070.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-22T07:01:48.328 INFO:teuthology.orchestra.run.smithi070.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-22T07:01:48.387 INFO:teuthology.orchestra.run.smithi070.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-22T07:01:48.445 INFO:teuthology.orchestra.run.smithi070.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-04-22T07:01:48.504 INFO:teuthology.orchestra.run.smithi070.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-22T07:01:48.563 INFO:teuthology.orchestra.run.smithi070.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-22T07:01:48.621 INFO:teuthology.orchestra.run.smithi070.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-22T07:01:48.671 INFO:teuthology.orchestra.run.smithi070.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-22T07:01:48.730 INFO:teuthology.orchestra.run.smithi070.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 920 kB/s | 54 kB 00:00 2024-04-22T07:01:48.847 INFO:teuthology.orchestra.run.smithi070.stdout:(90/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.4 MB/s | 5.1 MB 00:00 2024-04-22T07:01:48.898 INFO:teuthology.orchestra.run.smithi070.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-04-22T07:01:48.956 INFO:teuthology.orchestra.run.smithi070.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 760 kB/s | 44 kB 00:00 2024-04-22T07:01:49.006 INFO:teuthology.orchestra.run.smithi070.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-22T07:01:49.065 INFO:teuthology.orchestra.run.smithi070.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-04-22T07:01:49.115 INFO:teuthology.orchestra.run.smithi070.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 952 kB/s | 47 kB 00:00 2024-04-22T07:01:49.199 INFO:teuthology.orchestra.run.smithi070.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-22T07:01:49.249 INFO:teuthology.orchestra.run.smithi070.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-22T07:01:49.310 INFO:teuthology.orchestra.run.smithi070.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.5 MB/s | 94 kB 00:00 2024-04-22T07:01:49.368 INFO:teuthology.orchestra.run.smithi070.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-22T07:01:49.427 INFO:teuthology.orchestra.run.smithi070.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-04-22T07:01:49.511 INFO:teuthology.orchestra.run.smithi070.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-04-22T07:01:49.653 INFO:teuthology.orchestra.run.smithi070.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-04-22T07:01:49.878 INFO:teuthology.orchestra.run.smithi070.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-04-22T07:01:49.978 INFO:teuthology.orchestra.run.smithi070.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-04-22T07:01:50.020 INFO:teuthology.orchestra.run.smithi070.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 467 kB/s | 19 kB 00:00 2024-04-22T07:01:50.262 INFO:teuthology.orchestra.run.smithi070.stdout:(106/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-04-22T07:01:50.354 INFO:teuthology.orchestra.run.smithi070.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-04-22T07:01:50.421 INFO:teuthology.orchestra.run.smithi070.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 728 kB/s | 49 kB 00:00 2024-04-22T07:01:50.654 INFO:teuthology.orchestra.run.smithi044.stdout:(8/150): ceph-common-19.0.0-3156.g86f7587a.el9. 3.1 MB/s | 21 MB 00:06 2024-04-22T07:01:50.720 INFO:teuthology.orchestra.run.smithi044.stdout:(9/150): ceph-selinux-19.0.0-3156.g86f7587a.el9 380 kB/s | 25 kB 00:00 2024-04-22T07:01:50.855 INFO:teuthology.orchestra.run.smithi070.stdout:(109/150): ceph-test-19.0.0-3156.g86f7587a.el9. 3.3 MB/s | 49 MB 00:15 2024-04-22T07:01:50.880 INFO:teuthology.orchestra.run.smithi070.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 147 kB/s | 67 kB 00:00 2024-04-22T07:01:50.947 INFO:teuthology.orchestra.run.smithi070.stdout:(111/150): libarrow-9.0.0-10.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2024-04-22T07:01:51.021 INFO:teuthology.orchestra.run.smithi044.stdout:(10/150): ceph-osd-19.0.0-3156.g86f7587a.el9.x8 4.4 MB/s | 17 MB 00:03 2024-04-22T07:01:51.047 INFO:teuthology.orchestra.run.smithi070.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-22T07:01:51.072 INFO:teuthology.orchestra.run.smithi070.stdout:(113/150): luarocks-3.9.2-1.el9.noarch.rpm 696 kB/s | 151 kB 00:00 2024-04-22T07:01:51.096 INFO:teuthology.orchestra.run.smithi044.stdout:(11/150): cephfs-mirror-19.0.0-3156.g86f7587a.e 2.9 MB/s | 225 kB 00:00 2024-04-22T07:01:51.098 INFO:teuthology.orchestra.run.smithi070.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-04-22T07:01:51.131 INFO:teuthology.orchestra.run.smithi070.stdout:(115/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-04-22T07:01:51.154 INFO:teuthology.orchestra.run.smithi044.stdout:(12/150): cephfs-shell-19.0.0-3156.g86f7587a.el 464 kB/s | 27 kB 00:00 2024-04-22T07:01:51.156 INFO:teuthology.orchestra.run.smithi070.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-22T07:01:51.182 INFO:teuthology.orchestra.run.smithi070.stdout:(117/150): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-22T07:01:51.207 INFO:teuthology.orchestra.run.smithi070.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-22T07:01:51.221 INFO:teuthology.orchestra.run.smithi044.stdout:(13/150): libcephfs-devel-19.0.0-3156.g86f7587a 477 kB/s | 32 kB 00:00 2024-04-22T07:01:51.249 INFO:teuthology.orchestra.run.smithi070.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 847 kB/s | 35 kB 00:00 2024-04-22T07:01:51.274 INFO:teuthology.orchestra.run.smithi070.stdout:(120/150): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-22T07:01:51.299 INFO:teuthology.orchestra.run.smithi070.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 3.9 MB/s | 203 kB 00:00 2024-04-22T07:01:51.333 INFO:teuthology.orchestra.run.smithi070.stdout:(122/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-04-22T07:01:51.367 INFO:teuthology.orchestra.run.smithi070.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-22T07:01:51.383 INFO:teuthology.orchestra.run.smithi070.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-22T07:01:51.396 INFO:teuthology.orchestra.run.smithi044.stdout:(14/150): libcephfs2-19.0.0-3156.g86f7587a.el9. 4.2 MB/s | 747 kB 00:00 2024-04-22T07:01:51.434 INFO:teuthology.orchestra.run.smithi070.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-22T07:01:51.451 INFO:teuthology.orchestra.run.smithi070.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 345 kB/s | 23 kB 00:00 2024-04-22T07:01:51.463 INFO:teuthology.orchestra.run.smithi044.stdout:(15/150): libcephsqlite-19.0.0-3156.g86f7587a.e 2.4 MB/s | 164 kB 00:00 2024-04-22T07:01:51.485 INFO:teuthology.orchestra.run.smithi070.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-04-22T07:01:51.502 INFO:teuthology.orchestra.run.smithi070.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-04-22T07:01:51.539 INFO:teuthology.orchestra.run.smithi044.stdout:(16/150): librados-devel-19.0.0-3156.g86f7587a. 1.7 MB/s | 127 kB 00:00 2024-04-22T07:01:51.561 INFO:teuthology.orchestra.run.smithi070.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-04-22T07:01:51.611 INFO:teuthology.orchestra.run.smithi070.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-22T07:01:51.661 INFO:teuthology.orchestra.run.smithi070.stdout:(131/150): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-04-22T07:01:51.673 INFO:teuthology.orchestra.run.smithi044.stdout:(17/150): libradosstriper1-19.0.0-3156.g86f7587 3.7 MB/s | 500 kB 00:00 2024-04-22T07:01:51.686 INFO:teuthology.orchestra.run.smithi070.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-04-22T07:01:51.728 INFO:teuthology.orchestra.run.smithi070.stdout:(133/150): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-04-22T07:01:51.753 INFO:teuthology.orchestra.run.smithi070.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-04-22T07:01:51.778 INFO:teuthology.orchestra.run.smithi070.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-22T07:01:51.812 INFO:teuthology.orchestra.run.smithi070.stdout:(136/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-22T07:01:51.838 INFO:teuthology.orchestra.run.smithi070.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 336 kB/s | 28 kB 00:00 2024-04-22T07:01:51.863 INFO:teuthology.orchestra.run.smithi070.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-04-22T07:01:51.888 INFO:teuthology.orchestra.run.smithi070.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-22T07:01:51.913 INFO:teuthology.orchestra.run.smithi070.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-22T07:01:51.938 INFO:teuthology.orchestra.run.smithi070.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-22T07:01:51.964 INFO:teuthology.orchestra.run.smithi070.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-22T07:01:51.989 INFO:teuthology.orchestra.run.smithi070.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-22T07:01:52.031 INFO:teuthology.orchestra.run.smithi070.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 482 kB/s | 20 kB 00:00 2024-04-22T07:01:52.056 INFO:teuthology.orchestra.run.smithi070.stdout:(145/150): python3-websocket-client-1.2.3-2.el9 763 kB/s | 90 kB 00:00 2024-04-22T07:01:52.081 INFO:teuthology.orchestra.run.smithi070.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-22T07:01:52.107 INFO:teuthology.orchestra.run.smithi070.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-22T07:01:52.541 INFO:teuthology.orchestra.run.smithi070.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-04-22T07:01:52.874 INFO:teuthology.orchestra.run.smithi044.stdout:(18/150): librgw2-19.0.0-3156.g86f7587a.el9.x86 4.5 MB/s | 5.4 MB 00:01 2024-04-22T07:01:52.957 INFO:teuthology.orchestra.run.smithi044.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.g86 543 kB/s | 45 kB 00:00 2024-04-22T07:01:53.049 INFO:teuthology.orchestra.run.smithi070.stdout:(149/150): librbd1-19.0.0-3156.g86f7587a.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-04-22T07:01:53.058 INFO:teuthology.orchestra.run.smithi044.stdout:(20/150): python3-ceph-common-19.0.0-3156.g86f7 1.3 MB/s | 135 kB 00:00 2024-04-22T07:01:53.141 INFO:teuthology.orchestra.run.smithi044.stdout:(21/150): python3-cephfs-19.0.0-3156.g86f7587a. 1.9 MB/s | 162 kB 00:00 2024-04-22T07:01:53.274 INFO:teuthology.orchestra.run.smithi070.stdout:(150/150): librados2-19.0.0-3156.g86f7587a.el9. 2.8 MB/s | 3.4 MB 00:01 2024-04-22T07:01:53.277 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-04-22T07:01:53.277 INFO:teuthology.orchestra.run.smithi070.stdout:Total 5.5 MB/s | 218 MB 00:39 2024-04-22T07:01:53.283 INFO:teuthology.orchestra.run.smithi044.stdout:(22/150): python3-rados-19.0.0-3156.g86f7587a.e 2.2 MB/s | 320 kB 00:00 2024-04-22T07:01:53.367 INFO:teuthology.orchestra.run.smithi044.stdout:(23/150): python3-rbd-19.0.0-3156.g86f7587a.el9 3.5 MB/s | 298 kB 00:00 2024-04-22T07:01:53.450 INFO:teuthology.orchestra.run.smithi044.stdout:(24/150): python3-rgw-19.0.0-3156.g86f7587a.el9 1.2 MB/s | 100 kB 00:00 2024-04-22T07:01:53.526 INFO:teuthology.orchestra.run.smithi044.stdout:(25/150): rbd-fuse-19.0.0-3156.g86f7587a.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-22T07:01:53.960 INFO:teuthology.orchestra.run.smithi044.stdout:(26/150): ceph-radosgw-19.0.0-3156.g86f7587a.el 1.8 MB/s | 11 MB 00:05 2024-04-22T07:01:54.126 INFO:teuthology.orchestra.run.smithi044.stdout:(27/150): rbd-nbd-19.0.0-3156.g86f7587a.el9.x86 1.0 MB/s | 171 kB 00:00 2024-04-22T07:01:54.210 INFO:teuthology.orchestra.run.smithi044.stdout:(28/150): rbd-mirror-19.0.0-3156.g86f7587a.el9. 4.5 MB/s | 3.1 MB 00:00 2024-04-22T07:01:54.235 INFO:teuthology.orchestra.run.smithi044.stdout:(29/150): ceph-grafana-dashboards-19.0.0-3156.g 315 kB/s | 34 kB 00:00 2024-04-22T07:01:54.285 INFO:teuthology.orchestra.run.smithi044.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3156.g86f7587 1.9 MB/s | 142 kB 00:00 2024-04-22T07:01:54.349 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-04-22T07:01:54.472 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-04-22T07:01:54.473 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-04-22T07:01:55.219 INFO:teuthology.orchestra.run.smithi044.stdout:(31/150): ceph-mgr-diskprediction-local-19.0.0- 7.9 MB/s | 7.4 MB 00:00 2024-04-22T07:01:55.277 INFO:teuthology.orchestra.run.smithi044.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.g86 4.2 MB/s | 249 kB 00:00 2024-04-22T07:01:55.328 INFO:teuthology.orchestra.run.smithi044.stdout:(33/150): ceph-mgr-rook-19.0.0-3156.g86f7587a.e 987 kB/s | 49 kB 00:00 2024-04-22T07:01:55.395 INFO:teuthology.orchestra.run.smithi044.stdout:(34/150): ceph-mgr-dashboard-19.0.0-3156.g86f75 3.1 MB/s | 3.5 MB 00:01 2024-04-22T07:01:55.420 INFO:teuthology.orchestra.run.smithi044.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3156.g8 179 kB/s | 16 kB 00:00 2024-04-22T07:01:55.495 INFO:teuthology.orchestra.run.smithi044.stdout:(36/150): ceph-volume-19.0.0-3156.g86f7587a.el9 2.7 MB/s | 273 kB 00:00 2024-04-22T07:01:55.529 INFO:teuthology.orchestra.run.smithi044.stdout:(37/150): cephadm-19.0.0-3156.g86f7587a.el9.noa 6.9 MB/s | 763 kB 00:00 2024-04-22T07:01:55.554 INFO:teuthology.orchestra.run.smithi044.stdout:(38/150): cephfs-top-19.0.0-3156.g86f7587a.el9. 312 kB/s | 18 kB 00:00 2024-04-22T07:01:55.630 INFO:teuthology.orchestra.run.smithi044.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 804 kB/s | 80 kB 00:00 2024-04-22T07:01:55.696 INFO:teuthology.orchestra.run.smithi044.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-22T07:01:55.763 INFO:teuthology.orchestra.run.smithi044.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-04-22T07:01:55.788 INFO:teuthology.orchestra.run.smithi044.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-22T07:01:55.841 INFO:teuthology.orchestra.run.smithi044.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-22T07:01:55.866 INFO:teuthology.orchestra.run.smithi044.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-22T07:01:55.892 INFO:teuthology.orchestra.run.smithi044.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-22T07:01:55.958 INFO:teuthology.orchestra.run.smithi044.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-22T07:01:55.984 INFO:teuthology.orchestra.run.smithi044.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-22T07:01:56.180 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-04-22T07:01:56.181 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-04-22T07:01:56.192 INFO:teuthology.orchestra.run.smithi044.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-04-22T07:01:56.259 INFO:teuthology.orchestra.run.smithi044.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-04-22T07:01:56.284 INFO:teuthology.orchestra.run.smithi044.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-04-22T07:01:56.310 INFO:teuthology.orchestra.run.smithi044.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-22T07:01:56.344 INFO:teuthology.orchestra.run.smithi044.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-22T07:01:56.394 INFO:teuthology.orchestra.run.smithi044.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 392 kB/s | 33 kB 00:00 2024-04-22T07:01:56.453 INFO:teuthology.orchestra.run.smithi044.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-22T07:01:56.519 INFO:teuthology.orchestra.run.smithi044.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-22T07:01:56.586 INFO:teuthology.orchestra.run.smithi044.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-22T07:01:56.645 INFO:teuthology.orchestra.run.smithi044.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 779 kB/s | 45 kB 00:00 2024-04-22T07:01:56.745 INFO:teuthology.orchestra.run.smithi044.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-22T07:01:56.821 INFO:teuthology.orchestra.run.smithi044.stdout:(59/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.3 MB/s | 3.0 MB 00:00 2024-04-22T07:01:56.871 INFO:teuthology.orchestra.run.smithi044.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-04-22T07:01:56.920 INFO:teuthology.orchestra.run.smithi044.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 256 kB/s | 25 kB 00:00 2024-04-22T07:01:56.947 INFO:teuthology.orchestra.run.smithi044.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-04-22T07:01:56.980 INFO:teuthology.orchestra.run.smithi044.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 292 kB 00:00 2024-04-22T07:01:57.005 INFO:teuthology.orchestra.run.smithi044.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-22T07:01:57.031 INFO:teuthology.orchestra.run.smithi044.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-04-22T07:01:57.473 INFO:teuthology.orchestra.run.smithi044.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-04-22T07:01:57.539 INFO:teuthology.orchestra.run.smithi044.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 8.1 MB/s | 4.1 MB 00:00 2024-04-22T07:01:57.973 INFO:teuthology.orchestra.run.smithi044.stdout:(68/150): ceph-test-19.0.0-3156.g86f7587a.el9.x 6.8 MB/s | 49 MB 00:07 2024-04-22T07:01:58.015 INFO:teuthology.orchestra.run.smithi044.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 1.9 MB/s | 911 kB 00:00 2024-04-22T07:01:58.057 INFO:teuthology.orchestra.run.smithi044.stdout:(70/150): postgresql-13.14-1.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-04-22T07:01:58.158 INFO:teuthology.orchestra.run.smithi044.stdout:(71/150): postgresql-private-libs-13.14-1.el9.x 725 kB/s | 133 kB 00:00 2024-04-22T07:01:58.199 INFO:teuthology.orchestra.run.smithi044.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-04-22T07:01:58.225 INFO:teuthology.orchestra.run.smithi044.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-22T07:01:58.317 INFO:teuthology.orchestra.run.smithi044.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-22T07:01:58.525 INFO:teuthology.orchestra.run.smithi044.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 248 kB 00:00 2024-04-22T07:01:58.575 INFO:teuthology.orchestra.run.smithi044.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-04-22T07:01:58.617 INFO:teuthology.orchestra.run.smithi044.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-04-22T07:01:58.676 INFO:teuthology.orchestra.run.smithi044.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-22T07:01:58.760 INFO:teuthology.orchestra.run.smithi044.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 416 kB/s | 35 kB 00:00 2024-04-22T07:01:59.043 INFO:teuthology.orchestra.run.smithi044.stdout:(80/150): python3-babel-2.9.1-2.el9.noarch.rpm 7.1 MB/s | 6.0 MB 00:00 2024-04-22T07:01:59.119 INFO:teuthology.orchestra.run.smithi044.stdout:(81/150): postgresql-server-13.14-1.el9.x86_64. 5.2 MB/s | 5.8 MB 00:01 2024-04-22T07:01:59.144 INFO:teuthology.orchestra.run.smithi044.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-22T07:01:59.202 INFO:teuthology.orchestra.run.smithi044.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-22T07:01:59.228 INFO:teuthology.orchestra.run.smithi044.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-22T07:01:59.261 INFO:teuthology.orchestra.run.smithi044.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-04-22T07:01:59.320 INFO:teuthology.orchestra.run.smithi044.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-22T07:01:59.336 INFO:teuthology.orchestra.run.smithi044.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-22T07:01:59.387 INFO:teuthology.orchestra.run.smithi044.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-22T07:01:59.412 INFO:teuthology.orchestra.run.smithi044.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-22T07:01:59.446 INFO:teuthology.orchestra.run.smithi044.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 917 kB/s | 54 kB 00:00 2024-04-22T07:01:59.504 INFO:teuthology.orchestra.run.smithi044.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-04-22T07:01:59.554 INFO:teuthology.orchestra.run.smithi044.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-04-22T07:01:59.613 INFO:teuthology.orchestra.run.smithi044.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-22T07:01:59.680 INFO:teuthology.orchestra.run.smithi044.stdout:(94/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.5 MB/s | 5.1 MB 00:00 2024-04-22T07:01:59.705 INFO:teuthology.orchestra.run.smithi044.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 390 kB/s | 36 kB 00:00 2024-04-22T07:01:59.739 INFO:teuthology.orchestra.run.smithi044.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-22T07:01:59.798 INFO:teuthology.orchestra.run.smithi044.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-22T07:01:59.823 INFO:teuthology.orchestra.run.smithi044.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 693 kB/s | 58 kB 00:00 2024-04-22T07:01:59.856 INFO:teuthology.orchestra.run.smithi044.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-22T07:01:59.882 INFO:teuthology.orchestra.run.smithi044.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-22T07:01:59.916 INFO:teuthology.orchestra.run.smithi044.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 760 kB/s | 45 kB 00:00 2024-04-22T07:01:59.949 INFO:teuthology.orchestra.run.smithi044.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 331 kB/s | 22 kB 00:00 2024-04-22T07:02:00.092 INFO:teuthology.orchestra.run.smithi044.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-22T07:02:00.950 INFO:teuthology.orchestra.run.smithi044.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 360 kB/s | 309 kB 00:00 2024-04-22T07:02:01.083 INFO:teuthology.orchestra.run.smithi044.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-04-22T07:02:01.142 INFO:teuthology.orchestra.run.smithi044.stdout:(106/150): abseil-cpp-20211102.0-3.el9.x86_64.r 460 kB/s | 548 kB 00:01 2024-04-22T07:02:01.251 INFO:teuthology.orchestra.run.smithi044.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-22T07:02:01.359 INFO:teuthology.orchestra.run.smithi044.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-04-22T07:02:01.660 INFO:teuthology.orchestra.run.smithi044.stdout:(109/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.2 MB/s | 16 MB 00:02 2024-04-22T07:02:01.685 INFO:teuthology.orchestra.run.smithi044.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 207 kB/s | 67 kB 00:00 2024-04-22T07:02:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-04-22T07:02:02.309 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T07:02:02.351 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T07:02:02.385 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T07:02:02.435 INFO:teuthology.orchestra.run.smithi044.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 195 kB/s | 151 kB 00:00 2024-04-22T07:02:02.766 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T07:02:02.835 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:02:02.894 INFO:teuthology.orchestra.run.smithi044.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 694 kB/s | 838 kB 00:01 2024-04-22T07:02:02.956 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:02:02.994 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:03.011 INFO:teuthology.orchestra.run.smithi044.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-04-22T07:02:03.086 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:03.111 INFO:teuthology.orchestra.run.smithi044.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-04-22T07:02:03.127 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:02:03.157 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T07:02:03.183 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T07:02:03.211 INFO:teuthology.orchestra.run.smithi044.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-22T07:02:03.286 INFO:teuthology.orchestra.run.smithi044.stdout:(116/150): python3-asyncssh-2.13.2-5.el9.noarch 644 kB/s | 548 kB 00:00 2024-04-22T07:02:03.333 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T07:02:03.388 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T07:02:03.417 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:03.470 INFO:teuthology.orchestra.run.smithi044.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 664 kB/s | 172 kB 00:00 2024-04-22T07:02:03.503 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:03.526 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:02:03.588 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:02:03.610 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:02:03.632 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:02:03.795 INFO:teuthology.orchestra.run.smithi044.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-04-22T07:02:03.835 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:02:03.868 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:02:03.904 INFO:teuthology.orchestra.run.smithi044.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 699 kB/s | 303 kB 00:00 2024-04-22T07:02:03.929 INFO:teuthology.orchestra.run.smithi044.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 264 kB/s | 35 kB 00:00 2024-04-22T07:02:03.974 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T07:02:04.062 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T07:02:04.100 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T07:02:04.135 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T07:02:04.209 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T07:02:04.242 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T07:02:04.246 INFO:teuthology.orchestra.run.smithi044.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 593 kB/s | 203 kB 00:00 2024-04-22T07:02:04.286 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T07:02:04.319 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T07:02:04.346 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T07:02:04.416 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T07:02:04.477 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T07:02:04.507 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T07:02:04.513 INFO:teuthology.orchestra.run.smithi044.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 542 kB/s | 144 kB 00:00 2024-04-22T07:02:04.551 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T07:02:04.613 INFO:teuthology.orchestra.run.smithi044.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-22T07:02:04.627 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T07:02:04.662 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 30/152 2024-04-22T07:02:04.705 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 31/152 2024-04-22T07:02:04.713 INFO:teuthology.orchestra.run.smithi044.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-04-22T07:02:04.734 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 32/152 2024-04-22T07:02:04.804 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T07:02:04.814 INFO:teuthology.orchestra.run.smithi044.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-04-22T07:02:04.849 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T07:02:04.914 INFO:teuthology.orchestra.run.smithi044.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-22T07:02:04.933 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T07:02:05.014 INFO:teuthology.orchestra.run.smithi044.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-22T07:02:05.066 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T07:02:05.118 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T07:02:05.648 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T07:02:05.731 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T07:02:05.759 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T07:02:05.801 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T07:02:05.840 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 42/152 2024-04-22T07:02:05.873 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T07:02:05.928 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T07:02:06.003 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T07:02:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T07:02:06.077 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T07:02:06.124 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T07:02:06.160 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T07:02:06.203 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T07:02:06.245 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T07:02:06.323 INFO:teuthology.orchestra.run.smithi044.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 799 kB/s | 1.0 MB 00:01 2024-04-22T07:02:06.383 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T07:02:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T07:02:06.432 INFO:teuthology.orchestra.run.smithi044.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2024-04-22T07:02:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T07:02:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T07:02:06.557 INFO:teuthology.orchestra.run.smithi044.stdout:(130/150): python3-grpcio-1.46.7-10.el9.x86_64. 794 kB/s | 2.0 MB 00:02 2024-04-22T07:02:06.591 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T07:02:06.607 INFO:teuthology.orchestra.run.smithi044.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 450 kB/s | 79 kB 00:00 2024-04-22T07:02:06.630 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T07:02:06.670 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T07:02:06.696 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T07:02:06.733 INFO:teuthology.orchestra.run.smithi044.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 329 kB/s | 58 kB 00:00 2024-04-22T07:02:06.745 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T07:02:06.833 INFO:teuthology.orchestra.run.smithi044.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-22T07:02:06.908 INFO:teuthology.orchestra.run.smithi044.stdout:(134/150): libarrow-9.0.0-10.el9.x86_64.rpm 777 kB/s | 4.4 MB 00:05 2024-04-22T07:02:06.967 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T07:02:07.008 INFO:teuthology.orchestra.run.smithi044.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-04-22T07:02:07.034 INFO:teuthology.orchestra.run.smithi044.stdout:(136/150): python3-pecan-1.4.2-3.el9.noarch.rpm 639 kB/s | 272 kB 00:00 2024-04-22T07:02:07.059 INFO:teuthology.orchestra.run.smithi044.stdout:(137/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 398 kB/s | 90 kB 00:00 2024-04-22T07:02:07.109 INFO:teuthology.orchestra.run.smithi044.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-22T07:02:07.148 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T07:02:07.186 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T07:02:07.209 INFO:teuthology.orchestra.run.smithi044.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 358 kB/s | 36 kB 00:00 2024-04-22T07:02:07.213 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T07:02:07.243 INFO:teuthology.orchestra.run.smithi044.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 322 kB/s | 59 kB 00:00 2024-04-22T07:02:07.248 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T07:02:07.292 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T07:02:07.310 INFO:teuthology.orchestra.run.smithi044.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-04-22T07:02:07.330 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T07:02:07.335 INFO:teuthology.orchestra.run.smithi044.stdout:(142/150): python3-routes-2.5.1-5.el9.noarch.rp 626 kB/s | 188 kB 00:00 2024-04-22T07:02:07.361 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T07:02:07.391 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T07:02:07.425 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T07:02:07.459 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T07:02:07.494 INFO:teuthology.orchestra.run.smithi044.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 488 kB/s | 90 kB 00:00 2024-04-22T07:02:07.501 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T07:02:07.535 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T07:02:07.594 INFO:teuthology.orchestra.run.smithi044.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-04-22T07:02:07.619 INFO:teuthology.orchestra.run.smithi044.stdout:(145/150): python3-webob-1.8.7-6.el9.noarch.rpm 610 kB/s | 230 kB 00:00 2024-04-22T07:02:07.623 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T07:02:07.674 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T07:02:07.705 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T07:02:07.752 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T07:02:07.853 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T07:02:07.861 INFO:teuthology.orchestra.run.smithi044.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 718 kB/s | 192 kB 00:00 2024-04-22T07:02:07.907 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T07:02:07.970 INFO:teuthology.orchestra.run.smithi044.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 673 kB/s | 427 kB 00:00 2024-04-22T07:02:08.237 INFO:teuthology.orchestra.run.smithi044.stdout:(148/150): librados2-19.0.0-3156.g86f7587a.el9. 9.1 MB/s | 3.4 MB 00:00 2024-04-22T07:02:08.648 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T07:02:08.695 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T07:02:08.728 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T07:02:08.737 INFO:teuthology.orchestra.run.smithi044.stdout:(149/150): librbd1-19.0.0-3156.g86f7587a.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-04-22T07:02:08.855 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T07:02:08.911 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T07:02:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T07:02:09.047 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T07:02:09.223 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T07:02:09.253 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T07:02:09.357 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T07:02:09.704 INFO:teuthology.orchestra.run.smithi044.stdout:(150/150): thrift-0.15.0-2.el9.x86_64.rpm 777 kB/s | 1.6 MB 00:02 2024-04-22T07:02:09.710 INFO:teuthology.orchestra.run.smithi044.stdout:-------------------------------------------------------------------------------- 2024-04-22T07:02:09.710 INFO:teuthology.orchestra.run.smithi044.stdout:Total 6.4 MB/s | 218 MB 00:33 2024-04-22T07:02:10.017 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T07:02:10.137 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T07:02:10.759 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction check 2024-04-22T07:02:10.897 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction check succeeded. 2024-04-22T07:02:10.897 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction test 2024-04-22T07:02:11.383 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T07:02:11.457 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T07:02:11.485 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T07:02:11.518 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T07:02:11.831 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T07:02:11.873 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:02:11.963 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:02:11.993 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 98/152 2024-04-22T07:02:12.029 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T07:02:12.510 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T07:02:12.542 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:02:12.612 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:02:12.635 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 102/152 2024-04-22T07:02:12.720 INFO:teuthology.orchestra.run.smithi044.stdout:Transaction test succeeded. 2024-04-22T07:02:12.721 INFO:teuthology.orchestra.run.smithi044.stdout:Running transaction 2024-04-22T07:02:14.818 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:14.856 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:15.397 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:15.759 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:02:15.759 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:02:15.759 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T07:02:15.760 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T07:02:15.760 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:15.760 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:02:18.733 INFO:teuthology.orchestra.run.smithi044.stdout: Preparing : 1/1 2024-04-22T07:02:18.799 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T07:02:18.826 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T07:02:18.867 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T07:02:19.213 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T07:02:19.324 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:02:19.432 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 5/152 2024-04-22T07:02:19.467 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:19.559 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:19.601 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:02:19.629 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T07:02:19.656 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T07:02:19.815 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T07:02:19.861 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T07:02:19.889 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:19.994 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:20.023 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:02:20.069 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 13/152 2024-04-22T07:02:20.082 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:02:20.106 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T07:02:20.275 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:02:20.307 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T07:02:20.418 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T07:02:20.516 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T07:02:20.548 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T07:02:20.583 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T07:02:20.662 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T07:02:20.699 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T07:02:20.752 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T07:02:20.784 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T07:02:20.811 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T07:02:20.885 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T07:02:20.935 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T07:02:20.964 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T07:02:21.008 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T07:02:21.089 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T07:02:21.118 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 30/152 2024-04-22T07:02:21.154 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 31/152 2024-04-22T07:02:21.182 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 32/152 2024-04-22T07:02:21.256 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T07:02:21.305 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T07:02:21.384 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T07:02:21.530 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T07:02:21.584 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T07:02:22.153 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T07:02:22.240 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T07:02:22.274 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T07:02:22.317 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T07:02:22.346 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 42/152 2024-04-22T07:02:22.379 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T07:02:22.436 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T07:02:22.511 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T07:02:22.547 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T07:02:22.583 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T07:02:22.631 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T07:02:22.666 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T07:02:22.710 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T07:02:22.752 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T07:02:22.897 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T07:02:22.948 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T07:02:22.978 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T07:02:23.020 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T07:02:23.119 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T07:02:23.153 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T07:02:23.194 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T07:02:23.227 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T07:02:23.251 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T07:02:23.322 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T07:02:23.460 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T07:02:23.501 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T07:02:23.537 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T07:02:23.571 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T07:02:23.616 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T07:02:23.654 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T07:02:23.693 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T07:02:23.722 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T07:02:23.748 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T07:02:23.782 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T07:02:23.824 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T07:02:23.858 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T07:02:23.949 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T07:02:23.997 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T07:02:24.028 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T07:02:24.076 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T07:02:24.189 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T07:02:24.248 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T07:02:25.025 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T07:02:25.100 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T07:02:25.278 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T07:02:25.460 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T07:02:25.519 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T07:02:25.609 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T07:02:25.645 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T07:02:25.838 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T07:02:25.868 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T07:02:25.948 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T07:02:26.642 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T07:02:26.763 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T07:02:26.939 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:02:26.939 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-04-22T07:02:26.939 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-04-22T07:02:26.939 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-04-22T07:02:26.939 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-04-22T07:02:26.940 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-04-22T07:02:26.940 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-04-22T07:02:26.940 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-04-22T07:02:26.940 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:27.130 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:02:27.146 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:02:27.147 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:27.147 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T07:02:27.147 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:28.083 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T07:02:28.155 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T07:02:28.184 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T07:02:28.216 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T07:02:28.545 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T07:02:28.595 INFO:teuthology.orchestra.run.smithi044.stdout: Upgrading : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:02:28.625 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:02:28.650 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:02:28.650 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:28.651 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T07:02:28.651 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:02:28.651 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:02:28.651 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:28.687 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 97/152 2024-04-22T07:02:28.717 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 98/152 2024-04-22T07:02:28.753 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T07:02:28.875 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:02:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:02:28.898 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:28.898 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T07:02:28.898 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:02:28.898 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:02:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:29.262 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T07:02:29.281 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:02:29.343 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:02:29.344 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 101/152 2024-04-22T07:02:29.365 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:02:29.365 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:29.365 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T07:02:29.365 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:02:29.366 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:02:29.366 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:29.383 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 102/152 2024-04-22T07:02:29.473 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T07:02:29.530 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T07:02:29.585 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T07:02:29.769 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T07:02:29.810 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T07:02:29.888 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T07:02:29.978 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T07:02:30.123 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T07:02:30.159 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T07:02:30.192 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T07:02:30.219 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T07:02:30.671 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:02:30.701 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:02:30.733 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 122/152 2024-04-22T07:02:30.796 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 123/152 2024-04-22T07:02:30.833 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:02:31.823 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:31.854 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:31.862 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:02:31.893 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:02:32.426 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 103/152 2024-04-22T07:02:32.467 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:02:32.558 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 104/152 2024-04-22T07:02:32.558 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T07:02:32.558 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T07:02:32.559 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:32.592 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:02:32.850 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:02:32.885 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:02:32.984 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:02:33.149 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 127/152 2024-04-22T07:02:33.178 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:02:33.202 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:02:33.202 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:33.202 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T07:02:33.202 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:02:33.202 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:02:33.202 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:33.246 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:02:33.257 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:02:33.355 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 130/152 2024-04-22T07:02:33.403 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 131/152 2024-04-22T07:02:34.325 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T07:02:34.357 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:02:34.379 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:02:34.380 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:34.380 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T07:02:34.380 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:02:34.380 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:02:34.380 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:02:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:02:34.446 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:34.446 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T07:02:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:02:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:02:34.513 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:34.513 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T07:02:34.514 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:02:34.514 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:02:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:34.817 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:02:34.839 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:02:34.839 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:34.839 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T07:02:34.840 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:02:34.840 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:02:34.840 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:39.436 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 137/152 2024-04-22T07:02:39.486 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 138/152 2024-04-22T07:02:39.526 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 139/152 2024-04-22T07:02:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T07:02:39.594 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T07:02:39.777 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 142/152 2024-04-22T07:02:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T07:02:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 144/152 2024-04-22T07:02:40.466 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:40.501 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:40.521 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:41.294 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 146/152 2024-04-22T07:02:41.381 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T07:02:41.421 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T07:02:41.502 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T07:02:41.535 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T07:02:41.535 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:02:41.619 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:02:41.619 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:02:44.555 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 105/152 2024-04-22T07:02:44.555 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /sys 2024-04-22T07:02:44.555 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /proc 2024-04-22T07:02:44.555 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /mnt 2024-04-22T07:02:44.555 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /var/tmp 2024-04-22T07:02:44.555 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /home 2024-04-22T07:02:44.555 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /root 2024-04-22T07:02:44.555 INFO:teuthology.orchestra.run.smithi044.stdout:skipping the directory /tmp 2024-04-22T07:02:44.555 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:44.754 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:02:44.770 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 106/152 2024-04-22T07:02:44.770 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:44.770 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T07:02:44.771 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:46.308 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:02:46.334 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 107/152 2024-04-22T07:02:46.334 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:46.334 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T07:02:46.335 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:02:46.335 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T07:02:46.335 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:46.585 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:02:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 108/152 2024-04-22T07:02:46.714 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:46.714 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T07:02:46.714 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:02:46.714 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T07:02:46.714 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:47.076 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:02:47.099 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 109/152 2024-04-22T07:02:47.099 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:47.099 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T07:02:47.099 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:02:47.100 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T07:02:47.100 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:47.207 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T07:02:47.258 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T07:02:47.337 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T07:02:47.533 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T07:02:47.577 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 1/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 3/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 4/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 5/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 8/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 9/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 10/152 2024-04-22T07:02:47.580 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 11/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 13/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 14/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 15/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 16/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 17/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 18/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 19/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 20/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 21/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 22/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 23/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 24/152 2024-04-22T07:02:47.581 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 25/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 26/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 27/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 28/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 29/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 30/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 31/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 32/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 33/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 34/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 35/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 36/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 37/152 2024-04-22T07:02:47.582 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 38/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 39/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T07:02:47.583 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T07:02:47.584 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T07:02:47.585 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T07:02:47.586 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T07:02:47.587 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T07:02:47.588 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T07:02:47.589 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T07:02:47.590 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T07:02:47.591 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T07:02:47.591 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T07:02:47.591 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T07:02:47.591 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 149/152 2024-04-22T07:02:47.591 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T07:02:47.591 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 151/152 2024-04-22T07:02:47.658 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T07:02:47.756 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T07:02:47.899 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T07:02:47.944 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T07:02:47.975 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T07:02:48.003 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T07:02:48.466 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:02:48.502 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 121/152 2024-04-22T07:02:48.543 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 122/152 2024-04-22T07:02:48.607 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 123/152 2024-04-22T07:02:48.633 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:02:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:02:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:49.705 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-04-22T07:02:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:02:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T07:02:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T07:02:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T07:02:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T07:02:49.709 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T07:02:49.710 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T07:02:49.710 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.710 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.710 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T07:02:49.710 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.710 INFO:teuthology.orchestra.run.smithi070.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T07:02:49.710 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T07:02:49.710 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T07:02:49.711 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T07:02:49.712 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T07:02:49.712 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T07:02:49.712 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T07:02:49.712 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T07:02:49.712 INFO:teuthology.orchestra.run.smithi070.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T07:02:49.712 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T07:02:49.712 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T07:02:49.712 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T07:02:49.712 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T07:02:49.713 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T07:02:49.714 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T07:02:49.714 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T07:02:49.714 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T07:02:49.714 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T07:02:49.714 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T07:02:49.714 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T07:02:49.714 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T07:02:49.714 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T07:02:49.715 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T07:02:49.715 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T07:02:49.715 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T07:02:49.715 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T07:02:49.715 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T07:02:49.715 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T07:02:49.715 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T07:02:49.715 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T07:02:49.715 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T07:02:49.715 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T07:02:49.716 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T07:02:49.717 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T07:02:49.717 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T07:02:49.717 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.717 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.717 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T07:02:49.717 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T07:02:49.717 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T07:02:49.717 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.717 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T07:02:49.717 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T07:02:49.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T07:02:49.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T07:02:49.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T07:02:49.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T07:02:49.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T07:02:49.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T07:02:49.718 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T07:02:49.719 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T07:02:49.719 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T07:02:49.719 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T07:02:49.719 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T07:02:49.719 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T07:02:49.719 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T07:02:49.719 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.719 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.719 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:02:49.719 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T07:02:49.720 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T07:02:49.720 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T07:02:49.720 INFO:teuthology.orchestra.run.smithi070.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T07:02:49.720 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T07:02:49.720 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T07:02:49.721 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T07:02:49.721 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:02:49.721 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-04-22T07:02:49.738 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 124/152 2024-04-22T07:02:49.777 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:02:50.057 DEBUG:teuthology.parallel:result is None 2024-04-22T07:02:50.798 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 125/152 2024-04-22T07:02:50.818 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:02:50.924 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 126/152 2024-04-22T07:02:51.098 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 127/152 2024-04-22T07:02:51.129 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:02:51.153 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 128/152 2024-04-22T07:02:51.153 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:51.153 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T07:02:51.153 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:02:51.153 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T07:02:51.153 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:51.205 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:02:51.215 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 129/152 2024-04-22T07:02:51.318 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 130/152 2024-04-22T07:02:51.371 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 131/152 2024-04-22T07:02:52.349 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T07:02:52.383 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:02:52.406 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 133/152 2024-04-22T07:02:52.407 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:52.407 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T07:02:52.407 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:02:52.407 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T07:02:52.407 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:52.450 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:02:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.g86f75 134/152 2024-04-22T07:02:52.473 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:52.473 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T07:02:52.473 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:52.517 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:02:52.540 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 135/152 2024-04-22T07:02:52.541 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:52.541 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T07:02:52.541 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:02:52.541 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T07:02:52.541 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:52.859 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:02:52.883 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 136/152 2024-04-22T07:02:52.883 INFO:teuthology.orchestra.run.smithi044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T07:02:52.883 INFO:teuthology.orchestra.run.smithi044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T07:02:52.883 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:02:52.883 INFO:teuthology.orchestra.run.smithi044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T07:02:52.883 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:02:57.764 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 137/152 2024-04-22T07:02:57.805 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 138/152 2024-04-22T07:02:57.846 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 139/152 2024-04-22T07:02:57.878 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T07:02:57.905 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T07:02:58.097 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 142/152 2024-04-22T07:02:58.142 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T07:02:58.164 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 144/152 2024-04-22T07:02:58.755 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:58.795 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:58.815 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T07:02:59.657 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 146/152 2024-04-22T07:02:59.741 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T07:02:59.783 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T07:02:59.866 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T07:02:59.895 INFO:teuthology.orchestra.run.smithi044.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T07:02:59.896 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:02:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T07:02:59.971 INFO:teuthology.orchestra.run.smithi044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 1/152 2024-04-22T07:03:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2/152 2024-04-22T07:03:06.216 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 3/152 2024-04-22T07:03:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 4/152 2024-04-22T07:03:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.g86f75 5/152 2024-04-22T07:03:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 6/152 2024-04-22T07:03:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 7/152 2024-04-22T07:03:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 8/152 2024-04-22T07:03:06.217 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 9/152 2024-04-22T07:03:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 10/152 2024-04-22T07:03:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 11/152 2024-04-22T07:03:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 12/152 2024-04-22T07:03:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 13/152 2024-04-22T07:03:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 14/152 2024-04-22T07:03:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_ 15/152 2024-04-22T07:03:06.218 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 16/152 2024-04-22T07:03:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 17/152 2024-04-22T07:03:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_6 18/152 2024-04-22T07:03:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86 19/152 2024-04-22T07:03:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 20/152 2024-04-22T07:03:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el 21/152 2024-04-22T07:03:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.g86f7587a.el9. 22/152 2024-04-22T07:03:06.219 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_6 23/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 24/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 25/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 26/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 27/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 28/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 29/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a. 30/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noa 31/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.n 32/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f 33/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el 34/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 35/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.e 36/152 2024-04-22T07:03:06.220 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 37/152 2024-04-22T07:03:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 38/152 2024-04-22T07:03:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 39/152 2024-04-22T07:03:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T07:03:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T07:03:06.221 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T07:03:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T07:03:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T07:03:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T07:03:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T07:03:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T07:03:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T07:03:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T07:03:06.222 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T07:03:06.223 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T07:03:06.224 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T07:03:06.225 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T07:03:06.226 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T07:03:06.227 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T07:03:06.228 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T07:03:06.229 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T07:03:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T07:03:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T07:03:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T07:03:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T07:03:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T07:03:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T07:03:06.230 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 149/152 2024-04-22T07:03:06.231 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T07:03:06.231 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 151/152 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout:Upgraded: 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: librados2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: librbd1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout:Installed: 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.319 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-base-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-grafana-dashboards-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-immutable-object-cache-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mds-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-dashboard-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-modules-core-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mgr-rook-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-mon-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.320 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-osd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-prometheus-alerts-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-radosgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-selinux-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-test-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: ceph-volume-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: cephadm-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-shell-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: cephfs-top-2:19.0.0-3156.g86f7587a.el9.noarch 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T07:03:08.321 INFO:teuthology.orchestra.run.smithi044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: libcephfs2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: libcephsqlite-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T07:03:08.322 INFO:teuthology.orchestra.run.smithi044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: librados-devel-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: libradosstriper1-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: librgw2-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T07:03:08.323 INFO:teuthology.orchestra.run.smithi044.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T07:03:08.324 INFO:teuthology.orchestra.run.smithi044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-argparse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ceph-common-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cephfs-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T07:03:08.325 INFO:teuthology.orchestra.run.smithi044.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T07:03:08.326 INFO:teuthology.orchestra.run.smithi044.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T07:03:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T07:03:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T07:03:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T07:03:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T07:03:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T07:03:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T07:03:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T07:03:08.327 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T07:03:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T07:03:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T07:03:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T07:03:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T07:03:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T07:03:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T07:03:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T07:03:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T07:03:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T07:03:08.328 INFO:teuthology.orchestra.run.smithi044.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rados-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rgw-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T07:03:08.329 INFO:teuthology.orchestra.run.smithi044.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-fuse-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-mirror-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: rbd-nbd-2:19.0.0-3156.g86f7587a.el9.x86_64 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T07:03:08.330 INFO:teuthology.orchestra.run.smithi044.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T07:03:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T07:03:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T07:03:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T07:03:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T07:03:08.331 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:08.331 INFO:teuthology.orchestra.run.smithi044.stdout:Complete! 2024-04-22T07:03:08.699 DEBUG:teuthology.parallel:result is None 2024-04-22T07:03:08.700 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:03:08.932 DEBUG:teuthology.orchestra.run.smithi044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T07:03:08.955 INFO:teuthology.orchestra.run.smithi044.stdout:19.0.0-3156.g86f7587a.el9 2024-04-22T07:03:08.955 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.g86f7587a.el9 2024-04-22T07:03:08.955 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.g86f7587a is installed. 2024-04-22T07:03:08.957 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:03:09.113 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T07:03:09.139 INFO:teuthology.orchestra.run.smithi070.stdout:19.0.0-3156.g86f7587a.el9 2024-04-22T07:03:09.140 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.g86f7587a.el9 2024-04-22T07:03:09.140 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.g86f7587a is installed. 2024-04-22T07:03:09.142 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-22T07:03:09.142 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:09.143 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T07:03:09.169 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:09.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T07:03:09.209 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-22T07:03:09.209 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:09.210 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T07:03:09.234 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T07:03:09.298 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:09.298 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T07:03:09.325 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T07:03:09.390 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-22T07:03:09.390 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:09.390 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T07:03:09.415 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T07:03:09.479 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:09.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T07:03:09.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T07:03:09.567 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-22T07:03:09.567 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:09.567 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T07:03:09.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T07:03:09.657 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:09.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T07:03:09.681 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T07:03:09.746 INFO:teuthology.run_tasks:Running task ceph... 2024-04-22T07:03:09.834 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-22T07:03:09.834 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 777 /var/log/ceph 2024-04-22T07:03:09.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /var/log/ceph 2024-04-22T07:03:09.859 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-22T07:03:09.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-22T07:03:09.901 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-22T07:03:09.924 INFO:tasks.ceph:Creating extra log directories... 2024-04-22T07:03:09.925 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-22T07:03:09.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-22T07:03:09.999 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-22T07:03:09.999 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-22T07:03:09.999 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-22T07:03:09.999 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:09.999 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-22T07:03:10.045 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:10.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:10.192 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:10.192 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-22T07:03:10.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:10.322 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:10.357 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-22T07:03:10.357 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-22T07:03:10.357 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157', 'branch': 'wip-lusov-quiescer', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/snaps}}', 'email': None, 'first_in_suite': False, 'job_id': '7668157', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-quiescer'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-quiescer', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 4362, 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'sleep_before_teardown': 0, 'subset': '222/111111', 'suite': 'fs', 'suite_branch': 'wip-lusov-quiescer', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi044.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJW4DH9/OuIrQPDDxiqj26fgsnmpfYkm+x/wb5NNR8Y5p1CnQ3FLcnYlHCQjc5E30uENfANhCb/KTxaQcwDa54w=', 'smithi070.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdaKEfSykVzVIZHdC406/XHcSmPSaAcpxjPhJfeCQCrhbd2qVl9nSXdPpBrNIGCrpVzfbG/iCeBBGWamxCjO4k='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': '86f7587a5a09af35f0895e1a2d08527638fae697', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'quiescer': None}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-22_05:52:53', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-22T07:03:10.358 DEBUG:teuthology.orchestra.run.smithi044:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-22T07:03:10.363 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-22T07:03:10.407 DEBUG:teuthology.orchestra.run.smithi044:> sudo install -d -m0777 -- /var/run/ceph 2024-04-22T07:03:10.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/run/ceph 2024-04-22T07:03:10.471 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:10.471 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T07:03:10.497 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T07:03:10.497 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_1 2024-04-22T07:03:10.554 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T07:03:10.554 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:10.554 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-22T07:03:10.554 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:10.554 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:10.554 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 07:03:00.465094331 +0000 2024-04-22T07:03:10.554 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 07:00:34.185055878 +0000 2024-04-22T07:03:10.554 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 07:00:34.185055878 +0000 2024-04-22T07:03:10.554 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 07:00:34.185055878 +0000 2024-04-22T07:03:10.555 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T07:03:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T07:03:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T07:03:10.616 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000248964 s, 2.1 MB/s 2024-04-22T07:03:10.618 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T07:03:10.674 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_2 2024-04-22T07:03:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T07:03:10.729 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:10.730 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-22T07:03:10.730 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:10.730 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:10.730 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 07:03:00.466094315 +0000 2024-04-22T07:03:10.730 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 07:00:34.547049883 +0000 2024-04-22T07:03:10.730 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 07:00:34.547049883 +0000 2024-04-22T07:03:10.730 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 07:00:34.547049883 +0000 2024-04-22T07:03:10.730 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T07:03:10.792 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T07:03:10.792 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T07:03:10.792 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000209418 s, 2.4 MB/s 2024-04-22T07:03:10.794 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T07:03:10.850 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_3 2024-04-22T07:03:10.905 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T07:03:10.905 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:10.905 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-22T07:03:10.905 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:10.906 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:10.906 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 07:03:00.466094315 +0000 2024-04-22T07:03:10.906 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 07:00:34.911043855 +0000 2024-04-22T07:03:10.906 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 07:00:34.911043855 +0000 2024-04-22T07:03:10.906 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 07:00:34.911043855 +0000 2024-04-22T07:03:10.906 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T07:03:10.968 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T07:03:10.968 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T07:03:10.968 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000200718 s, 2.6 MB/s 2024-04-22T07:03:10.970 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T07:03:11.027 DEBUG:teuthology.orchestra.run.smithi044:> stat /dev/vg_nvme/lv_4 2024-04-22T07:03:11.082 INFO:teuthology.orchestra.run.smithi044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T07:03:11.082 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:11.082 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-22T07:03:11.083 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:11.083 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:11.083 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 2024-04-22 07:03:00.466094315 +0000 2024-04-22T07:03:11.083 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 07:00:35.262038042 +0000 2024-04-22T07:03:11.083 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 07:00:35.262038042 +0000 2024-04-22T07:03:11.083 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: 2024-04-22 07:00:35.262038042 +0000 2024-04-22T07:03:11.083 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T07:03:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records in 2024-04-22T07:03:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:1+0 records out 2024-04-22T07:03:11.145 INFO:teuthology.orchestra.run.smithi044.stderr:512 bytes copied, 0.000255901 s, 2.0 MB/s 2024-04-22T07:03:11.146 DEBUG:teuthology.orchestra.run.smithi044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T07:03:11.202 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:11.203 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:11.203 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T07:03:11.220 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T07:03:11.221 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-04-22T07:03:11.276 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T07:03:11.276 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:11.276 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-22T07:03:11.276 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:11.277 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:11.277 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-22 07:02:42.114892194 +0000 2024-04-22T07:03:11.277 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-22 07:00:31.997329584 +0000 2024-04-22T07:03:11.277 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-22 07:00:31.997329584 +0000 2024-04-22T07:03:11.277 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-22 07:00:31.997329584 +0000 2024-04-22T07:03:11.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T07:03:11.340 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-22T07:03:11.340 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-22T07:03:11.341 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000113951 s, 4.5 MB/s 2024-04-22T07:03:11.342 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T07:03:11.399 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-04-22T07:03:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T07:03:11.455 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:11.455 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-22T07:03:11.455 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:11.455 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:11.455 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-22 07:02:42.114892194 +0000 2024-04-22T07:03:11.455 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-22 07:00:32.328319615 +0000 2024-04-22T07:03:11.455 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-22 07:00:32.328319615 +0000 2024-04-22T07:03:11.456 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-22 07:00:32.328319615 +0000 2024-04-22T07:03:11.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T07:03:11.518 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-22T07:03:11.518 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-22T07:03:11.519 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.00010247 s, 5.0 MB/s 2024-04-22T07:03:11.520 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T07:03:11.576 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-04-22T07:03:11.633 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T07:03:11.633 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:11.633 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-22T07:03:11.633 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:11.633 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:11.634 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-22 07:02:42.114892194 +0000 2024-04-22T07:03:11.634 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-22 07:00:32.646310037 +0000 2024-04-22T07:03:11.634 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-22 07:00:32.646310037 +0000 2024-04-22T07:03:11.634 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-22 07:00:32.646310037 +0000 2024-04-22T07:03:11.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T07:03:11.696 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-22T07:03:11.696 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-22T07:03:11.696 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000102588 s, 5.0 MB/s 2024-04-22T07:03:11.697 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T07:03:11.753 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-04-22T07:03:11.808 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T07:03:11.808 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T07:03:11.809 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-22T07:03:11.809 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:03:11.809 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T07:03:11.809 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-04-22 07:02:42.114892194 +0000 2024-04-22T07:03:11.809 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-04-22 07:00:32.975300128 +0000 2024-04-22T07:03:11.809 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-04-22 07:00:32.975300128 +0000 2024-04-22T07:03:11.809 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-04-22 07:00:32.975300128 +0000 2024-04-22T07:03:11.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T07:03:11.870 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-04-22T07:03:11.870 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-04-22T07:03:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000102827 s, 5.0 MB/s 2024-04-22T07:03:11.872 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T07:03:11.928 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:11.928 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-22T07:03:11.928 INFO:tasks.ceph:Generating config... 2024-04-22T07:03:11.929 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-22T07:03:11.929 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-04-22T07:03:11.930 INFO:tasks.ceph:[client] debug client = 20 2024-04-22T07:03:11.930 INFO:tasks.ceph:[client] debug ms = 1 2024-04-22T07:03:11.930 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-22T07:03:11.930 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-22T07:03:11.930 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-22T07:03:11.930 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-22T07:03:11.930 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-22T07:03:11.930 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-04-22T07:03:11.930 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-04-22T07:03:11.930 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-04-22T07:03:11.930 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-04-22T07:03:11.930 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-22T07:03:11.930 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-22T07:03:11.931 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-22T07:03:11.932 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-22T07:03:11.932 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-22T07:03:11.933 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-22T07:03:11.933 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-22T07:03:11.933 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-22T07:03:11.933 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-22T07:03:11.933 INFO:tasks.ceph:Setting up mon.a... 2024-04-22T07:03:11.933 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-22T07:03:12.006 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.keyring 2024-04-22T07:03:12.009 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-22T07:03:12.050 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T07:03:12.114 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.44'), ('mon.b', '172.21.15.70'), ('mon.c', '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]')] 2024-04-22T07:03:12.114 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.44,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-22T07:03:12.115 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:12.115 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-22T07:03:12.168 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.44 --add b 172.21.15.70 --addv c '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:12.255 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: generated fsid dc21e26c-6c65-477f-a367-ffb001b78b47 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:setting min_mon_release = quincy 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:epoch 0 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:fsid dc21e26c-6c65-477f-a367-ffb001b78b47 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:last_changed 2024-04-22T07:03:12.258752+0000 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:created 2024-04-22T07:03:12.258752+0000 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:min_mon_release 17 (quincy) 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:election_strategy: 1 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:0: [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] mon.a 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:1: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.b 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:2: [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] mon.c 2024-04-22T07:03:12.256 INFO:teuthology.orchestra.run.smithi044.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-22T07:03:12.258 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-22T07:03:12.273 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID dc21e26c-6c65-477f-a367-ffb001b78b47... 2024-04-22T07:03:12.274 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-22T07:03:12.316 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-22T07:03:12.355 INFO:teuthology.orchestra.run.smithi044.stdout:[global] 2024-04-22T07:03:12.355 INFO:teuthology.orchestra.run.smithi044.stdout: chdir = "" 2024-04-22T07:03:12.355 INFO:teuthology.orchestra.run.smithi044.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-22T07:03:12.355 INFO:teuthology.orchestra.run.smithi044.stdout: auth supported = cephx 2024-04-22T07:03:12.355 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.355 INFO:teuthology.orchestra.run.smithi044.stdout: filestore xattr use omap = true 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: mon clock drift allowed = 1.000 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: osd crush chooseleaf type = 0 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: auth debug = true 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on old message = true 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: ms die on bug = true 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: mon pg warn max object skew = 0 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: # disable pg_autoscaler by default for new pools 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-22T07:03:12.356 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default size = 2 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow primary affinity = true 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd allow pg remap = true 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on legacy crush tunables = false 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on crush straw calc version zero = false 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on no sortbitwise = false 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on osd down out interval zero = false 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon warn on too few osds = false 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-22T07:03:12.357 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: mon_allow_pool_size_one = true 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: osd default data pool replay window = 5 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: mon allow pool delete = true 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: mon cluster log file level = debug 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: debug asserts on shutdown = true 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: mon health detail to clog = false 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: mon host = "172.21.15.44,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: fsid = dc21e26c-6c65-477f-a367-ffb001b78b47 2024-04-22T07:03:12.358 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout:[osd] 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd journal size = 100 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub load threshold = 5.0 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd scrub max interval = 600 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd mclock profile = high_recovery_ops 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd recover clone overlap = true 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd recovery max chunk = 1048576 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug shutdown = true 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug op order = true 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug verify stray on activate = true 2024-04-22T07:03:12.359 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug trim objects = true 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: osd open classes on start = true 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug pg log writeout = true 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: osd deep scrub update digest min age = 30 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: osd map max advance = 10 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: journal zero on create = true 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: filestore ondisk finisher threads = 3 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: filestore apply finisher threads = 3 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: bdev debug aio = true 2024-04-22T07:03:12.360 INFO:teuthology.orchestra.run.smithi044.stdout: osd debug misdirected ops = true 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: bdev async discard = True 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: bdev enable discard = True 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore allocator = bitmap 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore block size = 96636764160 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: bluestore fsck on mount = True 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluefs = 1/20 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug bluestore = 1/20 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug osd = 20 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: debug rocksdb = 4/10 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd backfillfull_ratio = 0.85 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd full ratio = 0.9 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd nearfull ratio = 0.8 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: osd blocked scrub grace period = 3600 2024-04-22T07:03:12.361 INFO:teuthology.orchestra.run.smithi044.stdout: osd failsafe full ratio = 0.95 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: osd max markdown count = 1000 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: osd objectstore = bluestore 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: osd op complaint time = 180 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr] 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug mgr = 20 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: mgr/telemetry/nag = false 2024-04-22T07:03:12.362 INFO:teuthology.orchestra.run.smithi044.stdout: debug client = 20 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout:[mon] 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: debug mon = 20 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: debug paxos = 20 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: debug auth = 20 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon data avail warn = 5 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon mgr mkfs grace = 240 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min pgs per osd = 4 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd reporter subtree level = osd 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon osd prime pg temp = true 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: mon reweight min bytes per osd = 10 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-22T07:03:12.363 INFO:teuthology.orchestra.run.smithi044.stdout: auth mon ticket ttl = 660 # 11m 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: auth service ticket ttl = 240 # 4m 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: # don't complain about insecure global_id in the test suite 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: # 1m isn't quite enough 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: mon_down_mkfs_grace = 2m 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: mon_warn_on_filestore_osds = false 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: mon down mkfs grace = 300 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: mon op complaint time = 120 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:12.364 INFO:teuthology.orchestra.run.smithi044.stdout:[client] 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: rgw cache enabled = true 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable ops log = true 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: rgw enable usage log = true 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: client mount timeout = 600 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: client_shutdown_timeout = 120 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: debug client = 20 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: rados mon op timeout = 900 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout: rados osd op timeout = 900 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.a] 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.b] 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mon.c] 2024-04-22T07:03:12.365 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c] 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.e] 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b] 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d] 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.f] 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout:[mds] 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: debug mds = 20 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: debug mds balancer = 20 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: debug ms = 1 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal fragment size max = 10000 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal merge size = 5 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal split bits = 3 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: mds bal split size = 100 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: mds debug frag = True 2024-04-22T07:03:12.366 INFO:teuthology.orchestra.run.smithi044.stdout: mds debug scatterstat = True 2024-04-22T07:03:12.367 INFO:teuthology.orchestra.run.smithi044.stdout: mds oft prefetch dirfrags = True 2024-04-22T07:03:12.367 INFO:teuthology.orchestra.run.smithi044.stdout: mds op complaint time = 180 2024-04-22T07:03:12.367 INFO:teuthology.orchestra.run.smithi044.stdout: mds verify scatter = True 2024-04-22T07:03:12.367 INFO:teuthology.orchestra.run.smithi044.stdout: mds_dir_prefetch = True 2024-04-22T07:03:12.367 INFO:teuthology.orchestra.run.smithi044.stdout: mds_sleep_rank_change = 5000000.0 2024-04-22T07:03:12.367 INFO:teuthology.orchestra.run.smithi044.stdout: osd op complaint time = 180 2024-04-22T07:03:12.367 INFO:teuthology.orchestra.run.smithi044.stdout: rados mon op timeout = 900 2024-04-22T07:03:12.367 INFO:teuthology.orchestra.run.smithi044.stdout: rados osd op timeout = 900 2024-04-22T07:03:12.367 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2024-04-22T07:03:12.368 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2024-04-22T07:03:12.369 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: mon_allow_pool_size_one = true 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2024-04-22T07:03:12.370 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "172.21.15.44,172.21.15.70,[v2:172.21.15.70:3301,v1:172.21.15.70:6790]" 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = dc21e26c-6c65-477f-a367-ffb001b78b47 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: osd mclock profile = high_recovery_ops 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2024-04-22T07:03:12.371 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug trim objects = true 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2024-04-22T07:03:12.372 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: bdev async discard = True 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: bdev enable discard = True 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore allocator = bitmap 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 1/20 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 1/20 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 4/10 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2024-04-22T07:03:12.373 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: osd blocked scrub grace period = 3600 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 0.95 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: osd max markdown count = 1000 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = bluestore 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-04-22T07:03:12.374 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.375 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: # 1m isn't quite enough 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: mon_down_mkfs_grace = 2m 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_filestore_osds = false 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: mon down mkfs grace = 300 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: mon op complaint time = 120 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-04-22T07:03:12.376 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: client mount timeout = 600 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: client_shutdown_timeout = 120 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2024-04-22T07:03:12.377 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c] 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.e] 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d] 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.f] 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout:[mds] 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds = 20 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds balancer = 20 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal fragment size max = 10000 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal merge size = 5 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal split bits = 3 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: mds bal split size = 100 2024-04-22T07:03:12.378 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug frag = True 2024-04-22T07:03:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug scatterstat = True 2024-04-22T07:03:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: mds oft prefetch dirfrags = True 2024-04-22T07:03:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: mds op complaint time = 180 2024-04-22T07:03:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: mds verify scatter = True 2024-04-22T07:03:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: mds_dir_prefetch = True 2024-04-22T07:03:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: mds_sleep_rank_change = 5000000.0 2024-04-22T07:03:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2024-04-22T07:03:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2024-04-22T07:03:12.379 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2024-04-22T07:03:12.380 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-22T07:03:12.381 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-22T07:03:12.475 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-22T07:03:12.475 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:12.475 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-22T07:03:12.490 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:12.490 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-22T07:03:12.545 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:03:12.545 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:12.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-22T07:03:12.546 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T07:03:12.619 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:12.620 DEBUG:teuthology.orchestra.run.smithi044:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:12.674 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T07:03:12.675 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:12.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-22T07:03:12.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T07:03:12.708 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:12.708 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:12.763 INFO:tasks.ceph:Setting up mon nodes... 2024-04-22T07:03:12.763 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-22T07:03:12.764 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-22T07:03:12.813 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-22T07:03:12.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-22T07:03:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-22T07:03:12.878 INFO:tasks.ceph:Setting up mds nodes... 2024-04-22T07:03:12.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-22T07:03:12.927 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-22T07:03:12.930 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-22T07:03:12.954 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-22T07:03:13.044 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-22T07:03:13.046 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-22T07:03:13.070 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-04-22T07:03:13.160 INFO:teuthology.orchestra.run.smithi044.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-04-22T07:03:13.162 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-04-22T07:03:13.186 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-22T07:03:13.235 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-22T07:03:13.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-22T07:03:13.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-22T07:03:13.349 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-22T07:03:13.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-22T07:03:13.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-04-22T07:03:13.462 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-04-22T07:03:13.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-04-22T07:03:13.487 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-22T07:03:13.488 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-22T07:03:13.529 INFO:teuthology.orchestra.run.smithi044.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-22T07:03:13.541 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-22T07:03:13.541 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-22T07:03:13.541 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-22T07:03:13.606 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:13.606 INFO:tasks.ceph:role: osd.0 2024-04-22T07:03:13.606 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:03:13.606 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-22T07:03:13.678 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:13.678 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:13.678 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:13.678 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:13.678 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:13.678 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:13.678 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:13.679 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:13.679 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:13.679 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:14.203 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-22T07:03:14.205 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-22T07:03:14.205 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-22T07:03:14.354 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-22T07:03:14.387 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-22T07:03:14.453 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:14.453 INFO:tasks.ceph:role: osd.1 2024-04-22T07:03:14.453 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:03:14.453 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-22T07:03:14.525 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:14.525 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:14.525 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:14.526 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:14.526 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:14.526 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:14.526 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:14.966 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-22T07:03:14.968 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-22T07:03:14.968 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-22T07:03:15.056 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-22T07:03:15.086 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-22T07:03:15.150 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:15.150 INFO:tasks.ceph:role: osd.2 2024-04-22T07:03:15.150 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:03:15.150 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-22T07:03:15.222 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:15.222 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:15.222 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:15.223 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:15.223 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:15.223 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:15.223 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:15.711 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-22T07:03:15.713 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-22T07:03:15.713 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-22T07:03:15.847 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-22T07:03:15.875 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-22T07:03:15.940 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:15.940 INFO:tasks.ceph:role: osd.3 2024-04-22T07:03:15.940 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:03:15.940 DEBUG:teuthology.orchestra.run.smithi044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-22T07:03:16.012 INFO:teuthology.orchestra.run.smithi044.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:16.012 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:16.012 INFO:teuthology.orchestra.run.smithi044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:16.012 INFO:teuthology.orchestra.run.smithi044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:16.012 INFO:teuthology.orchestra.run.smithi044.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:16.012 INFO:teuthology.orchestra.run.smithi044.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:16.012 INFO:teuthology.orchestra.run.smithi044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:16.012 INFO:teuthology.orchestra.run.smithi044.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:16.013 INFO:teuthology.orchestra.run.smithi044.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:16.013 INFO:teuthology.orchestra.run.smithi044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:16.482 INFO:teuthology.orchestra.run.smithi044.stdout:Discarding blocks...Done. 2024-04-22T07:03:16.484 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi044.front.sepia.ceph.com -o noatime 2024-04-22T07:03:16.484 DEBUG:teuthology.orchestra.run.smithi044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-22T07:03:16.617 DEBUG:teuthology.orchestra.run.smithi044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-22T07:03:16.647 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:16.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:16.745+0000 7f853961d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-22T07:03:16.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:16.747+0000 7f853961d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-22T07:03:16.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:16.747+0000 7f853961d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T07:03:16.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:16.747+0000 7f853961d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T07:03:16.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:16.747+0000 7f853961d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T07:03:16.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:16.747+0000 7f853961d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-22T07:03:19.601 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T07:03:19.630 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:19.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:19.720+0000 7f0d6762c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-22T07:03:19.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:19.720+0000 7f0d6762c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-22T07:03:19.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:19.720+0000 7f0d6762c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T07:03:19.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:19.720+0000 7f0d6762c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T07:03:19.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:19.720+0000 7f0d6762c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T07:03:19.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:19.722+0000 7f0d6762c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-22T07:03:22.579 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T07:03:22.608 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:22.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:22.702+0000 7fa9b9ddc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-22T07:03:22.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:22.703+0000 7fa9b9ddc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-22T07:03:22.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:22.703+0000 7fa9b9ddc640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T07:03:22.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:22.703+0000 7fa9b9ddc640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T07:03:22.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:22.703+0000 7fa9b9ddc640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T07:03:22.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:22.703+0000 7fa9b9ddc640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-22T07:03:25.559 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T07:03:25.588 DEBUG:teuthology.orchestra.run.smithi044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:25.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:25.681+0000 7f71f43ef640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-22T07:03:25.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:25.681+0000 7f71f43ef640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-22T07:03:25.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:25.681+0000 7f71f43ef640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T07:03:25.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:25.681+0000 7f71f43ef640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T07:03:25.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:25.681+0000 7f71f43ef640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T07:03:25.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:25.682+0000 7f71f43ef640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-22T07:03:28.536 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T07:03:28.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-22T07:03:28.594 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:28.594 INFO:tasks.ceph:role: osd.4 2024-04-22T07:03:28.594 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T07:03:28.594 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-22T07:03:28.660 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:28.660 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:28.660 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:28.660 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:28.660 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:28.661 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:28.661 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:28.661 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:28.661 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:28.661 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:28.693 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-04-22T07:03:28.694 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-04-22T07:03:28.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-22T07:03:28.764 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-22T07:03:28.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-22T07:03:28.897 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:28.897 INFO:tasks.ceph:role: osd.5 2024-04-22T07:03:28.898 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T07:03:28.898 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-22T07:03:28.960 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:28.960 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:28.961 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:28.961 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:28.961 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:28.961 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:28.961 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:28.993 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-04-22T07:03:28.995 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-04-22T07:03:28.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-22T07:03:29.067 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-22T07:03:29.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-22T07:03:29.200 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:29.200 INFO:tasks.ceph:role: osd.6 2024-04-22T07:03:29.200 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T07:03:29.200 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-22T07:03:29.263 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:29.263 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:29.263 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:29.263 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:29.263 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:29.263 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:29.263 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:29.263 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:29.263 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:29.263 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:29.295 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-04-22T07:03:29.297 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-04-22T07:03:29.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-22T07:03:29.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-22T07:03:29.428 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-22T07:03:29.492 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T07:03:29.493 INFO:tasks.ceph:role: osd.7 2024-04-22T07:03:29.493 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T07:03:29.493 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-22T07:03:29.555 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T07:03:29.556 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T07:03:29.556 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T07:03:29.556 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T07:03:29.556 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T07:03:29.556 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2024-04-22T07:03:29.556 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T07:03:29.556 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T07:03:29.556 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T07:03:29.556 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T07:03:29.588 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2024-04-22T07:03:29.590 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2024-04-22T07:03:29.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-22T07:03:29.666 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-22T07:03:29.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:29.830+0000 7fd04a78e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-22T07:03:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:29.830+0000 7fd04a78e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-22T07:03:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:29.830+0000 7fd04a78e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T07:03:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:29.830+0000 7fd04a78e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T07:03:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:29.830+0000 7fd04a78e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T07:03:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:29.830+0000 7fd04a78e640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-22T07:03:32.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T07:03:32.694 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:32.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:32.784+0000 7f37b0e14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-22T07:03:32.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:32.784+0000 7f37b0e14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-22T07:03:32.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:32.784+0000 7f37b0e14640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T07:03:32.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:32.784+0000 7f37b0e14640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T07:03:32.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:32.784+0000 7f37b0e14640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T07:03:32.782 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:32.785+0000 7f37b0e14640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-22T07:03:35.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T07:03:35.653 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:35.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:35.746+0000 7fea3438e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-22T07:03:35.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:35.746+0000 7fea3438e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-22T07:03:35.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:35.746+0000 7fea3438e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T07:03:35.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:35.746+0000 7fea3438e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T07:03:35.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:35.746+0000 7fea3438e640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T07:03:35.744 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:35.746+0000 7fea3438e640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-22T07:03:38.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-22T07:03:38.619 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:38.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:38.709+0000 7f6193c2e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-22T07:03:38.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:38.709+0000 7f6193c2e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-22T07:03:38.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:38.709+0000 7f6193c2e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T07:03:38.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:38.709+0000 7f6193c2e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T07:03:38.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:38.709+0000 7f6193c2e640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T07:03:38.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:38.709+0000 7f6193c2e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-22T07:03:40.359 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:40.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:03:41.547 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-22T07:03:41.573 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-22T07:03:41.573 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:41.573 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-22T07:03:41.604 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:41.604 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-22T07:03:41.672 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:41.672 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-22T07:03:41.742 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:41.742 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-04-22T07:03:41.811 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:41.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-22T07:03:41.879 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:41.879 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-22T07:03:41.947 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:41.947 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-22T07:03:42.016 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:42.016 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-22T07:03:42.085 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:42.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-22T07:03:42.113 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:42.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-22T07:03:42.180 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:42.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-22T07:03:42.246 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:42.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-04-22T07:03:42.311 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:42.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-22T07:03:42.377 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:42.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-22T07:03:42.442 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:42.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-22T07:03:42.506 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:42.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-22T07:03:42.570 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:42.570 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-22T07:03:42.589 INFO:tasks.ceph:Adding keys to all mons... 2024-04-22T07:03:42.589 DEBUG:teuthology.orchestra.run.smithi044:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-22T07:03:42.632 DEBUG:teuthology.orchestra.run.smithi070:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-22T07:03:42.654 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.y] 2024-04-22T07:03:42.654 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCwCyZmIh2sMBAAgpmPt+ta+BL2fOYMYzmRDQ== 2024-04-22T07:03:42.654 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCwCyZmLSB9NxAAZa9tk+iStehl8i3AnOb2BQ== 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c] 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCxCyZmDubRAhAAvBOBiJEDBDHyhlwPIbUqiQ== 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.e] 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCxCyZmrum7CRAAhYqc/XeuQ1RHO2D9ym7jdQ== 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.0] 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQC0CyZmLr+QLBAAMI4RaV2hMq3RplQ6sxx3bg== 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.1] 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQC3CyZmQ6gIKxAAalK18o4CXtr57CHifUD2rQ== 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.2] 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQC6CyZmlvXxKRAA2+CxJ6ve0GB1Xeacz49BAA== 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.3] 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQC9CyZmx/GjKBAAhTwaTRROB328zLNZvsKx9w== 2024-04-22T07:03:42.655 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.x] 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCwCyZmVQBiNBAALzaT/HmGerWQ8qUECREVVw== 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCxCyZmhclADhAARp24fmW8/M5kHAw9YWMjzA== 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d] 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCxCyZm06b/FBAACin6cqPAYylU9hPeegvgOA== 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.f] 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCxCyZmooq+GxAAoVBJ0jSWWYlFzPMYcfwdKg== 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.4] 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDBCyZmlpiFMRAALuoO/qEjPmSJJvISkhy1Ug== 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.5] 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDECyZmBS3KLhAAMU1ELdf3ZY6HAGM2jq880w== 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.6] 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDHCyZmV2OALBAAMfIunnTlEED5c6ubal0uKA== 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.7] 2024-04-22T07:03:42.656 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQDKCyZmj99PKhAAXvPvSJnJL+u137PnA8/1YA== 2024-04-22T07:03:42.657 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2024-04-22T07:03:42.657 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCxCyZm8mW8HxAAQrB6pZ9k3FpJIj+ytTcaPw== 2024-04-22T07:03:42.657 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.y] 2024-04-22T07:03:42.657 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCwCyZmIh2sMBAAgpmPt+ta+BL2fOYMYzmRDQ== 2024-04-22T07:03:42.657 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.a] 2024-04-22T07:03:42.657 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCwCyZmLSB9NxAAZa9tk+iStehl8i3AnOb2BQ== 2024-04-22T07:03:42.657 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.c] 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCxCyZmDubRAhAAvBOBiJEDBDHyhlwPIbUqiQ== 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.e] 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCxCyZmrum7CRAAhYqc/XeuQ1RHO2D9ym7jdQ== 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.0] 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQC0CyZmLr+QLBAAMI4RaV2hMq3RplQ6sxx3bg== 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.1] 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQC3CyZmQ6gIKxAAalK18o4CXtr57CHifUD2rQ== 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.2] 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQC6CyZmlvXxKRAA2+CxJ6ve0GB1Xeacz49BAA== 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.3] 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQC9CyZmx/GjKBAAhTwaTRROB328zLNZvsKx9w== 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout:[mgr.x] 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCwCyZmVQBiNBAALzaT/HmGerWQ8qUECREVVw== 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.b] 2024-04-22T07:03:42.658 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCxCyZmhclADhAARp24fmW8/M5kHAw9YWMjzA== 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.d] 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCxCyZm06b/FBAACin6cqPAYylU9hPeegvgOA== 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout:[mds.f] 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCxCyZmooq+GxAAoVBJ0jSWWYlFzPMYcfwdKg== 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.4] 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDBCyZmlpiFMRAALuoO/qEjPmSJJvISkhy1Ug== 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.5] 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDECyZmBS3KLhAAMU1ELdf3ZY6HAGM2jq880w== 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.6] 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDHCyZmV2OALBAAMfIunnTlEED5c6ubal0uKA== 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout:[osd.7] 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQDKCyZmj99PKhAAXvPvSJnJL+u137PnA8/1YA== 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout:[client.0] 2024-04-22T07:03:42.659 INFO:teuthology.orchestra.run.smithi044.stdout: key = AQCxCyZm8mW8HxAAQrB6pZ9k3FpJIj+ytTcaPw== 2024-04-22T07:03:42.661 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T07:03:42.703 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T07:03:42.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:42.760 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:42.814 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:42.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:42.868 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:42.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:42.922 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:42.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:42.973 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:42.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.024 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.026 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.075 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.077 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.125 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T07:03:43.128 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T07:03:43.177 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:43.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:43.231 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:43.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:43.285 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:43.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-22T07:03:43.342 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.393 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.395 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.444 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.495 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T07:03:43.546 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T07:03:43.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T07:03:43.600 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-22T07:03:43.600 DEBUG:teuthology.orchestra.run.smithi044:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-22T07:03:43.624 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T07:03:43.721 DEBUG:teuthology.orchestra.run.smithi044:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-22T07:03:43.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-22T07:03:43.769 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T07:03:43.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-22T07:03:43.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-22T07:03:43.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T07:03:44.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-22T07:03:44.066 DEBUG:teuthology.orchestra.run.smithi044:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:44.068 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-22T07:03:44.121 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-22T07:03:44.121 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-22T07:03:44.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-22T07:03:44.124 INFO:tasks.ceph.mon.a:Started 2024-04-22T07:03:44.125 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-22T07:03:44.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-22T07:03:44.164 INFO:tasks.ceph.mon.b:Started 2024-04-22T07:03:44.164 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-22T07:03:44.164 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-22T07:03:44.166 INFO:tasks.ceph.mon.c:Started 2024-04-22T07:03:44.167 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-22T07:03:44.167 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-22T07:03:44.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-22T07:03:44.169 INFO:tasks.ceph.mgr.y:Started 2024-04-22T07:03:44.169 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-22T07:03:44.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-22T07:03:44.171 INFO:tasks.ceph.mgr.x:Started 2024-04-22T07:03:44.171 INFO:tasks.ceph:Setting crush tunables to default 2024-04-22T07:03:44.171 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --cluster ceph osd crush tunables default 2024-04-22T07:03:44.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.276+0000 7f8100e55640 1 Processor -- start 2024-04-22T07:03:44.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.276+0000 7f8100e55640 1 -- start start 2024-04-22T07:03:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.277+0000 7f8100e55640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.277+0000 7f8100e55640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.277+0000 7f8100e55640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80fc0c7810 con 0x7f80fc154f20 2024-04-22T07:03:44.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.277+0000 7f8100e55640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f80fc0cb4b0 con 0x7f80fc15ac10 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.277+0000 7f8100e55640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f80fc0cd5e0 con 0x7f80fc0cd070 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.277+0000 7f80f9d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.277+0000 7f80f9d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46744/0 (socket says 172.21.15.44:46744) 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.278+0000 7f80f9d9b640 1 -- 172.21.15.44:0/1301323835 learned_addr learned my addr 172.21.15.44:0/1301323835 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.278+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.278+0000 7f80f9d9b640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 msgr2=0x7f80fc0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.278+0000 7f80f9d9b640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 msgr2=0x7f80fc0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.278+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.278+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.278+0000 7f80fad9d640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 msgr2=0x7f80fc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.278+0000 7f80fad9d640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 msgr2=0x7f80fc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.278+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:44.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.278+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:03:44.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.479+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:44.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.479+0000 7f80f9d9b640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 msgr2=0x7f80fc0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T07:03:44.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.479+0000 7f80f9d9b640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 msgr2=0x7f80fc0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:44.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.479+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:44.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.479+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:03:44.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.479+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:44.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.479+0000 7f80fad9d640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 msgr2=0x7f80fc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T07:03:44.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.479+0000 7f80fad9d640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 msgr2=0x7f80fc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:44.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.479+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:44.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.479+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:03:44.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.880+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:44.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.880+0000 7f80f9d9b640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 msgr2=0x7f80fc0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T07:03:44.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.880+0000 7f80f9d9b640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 msgr2=0x7f80fc0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:44.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.880+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:44.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.880+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:03:44.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.880+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:44.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.880+0000 7f80fad9d640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 msgr2=0x7f80fc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T07:03:44.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.880+0000 7f80fad9d640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 msgr2=0x7f80fc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:44.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.880+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:44.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:44.880+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:03:45.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:45.681+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:45.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:45.681+0000 7f80f9d9b640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 msgr2=0x7f80fc0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-22T07:03:45.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:45.681+0000 7f80f9d9b640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 msgr2=0x7f80fc0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:45.681+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:45.681+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:03:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:45.681+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:45.681+0000 7f80fad9d640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 msgr2=0x7f80fc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-22T07:03:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:45.681+0000 7f80fad9d640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 msgr2=0x7f80fc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:45.681+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:45.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:45.681+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:03:47.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.277+0000 7f80f3fff640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 msgr2=0x7f80fc0d03a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:03:47.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.277+0000 7f80f3fff640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80fc0cd070 0x7f80fc0d03a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.277+0000 7f80f3fff640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 msgr2=0x7f80fc0ccb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:03:47.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.277+0000 7f80f3fff640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.44:3300/0 conn(0x7f80fc15ac10 0x7f80fc0ccb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:47.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.277+0000 7f80f3fff640 1 -- 172.21.15.44:0/1301323835 >> v1:172.21.15.70:6789/0 conn(0x7f80fc154f20 legacy=0x7f80fc0cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:47.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.277+0000 7f80f3fff640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:47.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.278+0000 7f80f3fff640 1 -- 172.21.15.44:0/1301323835 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80fc0cd8c0 con 0x7f80dc006200 2024-04-22T07:03:47.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.278+0000 7f80f3fff640 1 -- 172.21.15.44:0/1301323835 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f80fc0cb4b0 con 0x7f80dc0036c0 2024-04-22T07:03:47.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.278+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.278+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 msgr2=0x7f80dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:03:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.278+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 msgr2=0x7f80dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.278+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:47.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.278+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:03:47.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.479+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.479+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 msgr2=0x7f80dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:03:47.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.479+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 msgr2=0x7f80dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:47.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.479+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:47.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.479+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:03:47.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.880+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:47.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.880+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 msgr2=0x7f80dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:03:47.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.880+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 msgr2=0x7f80dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:47.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.880+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:47.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:47.880+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:03:48.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:48.681+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:48.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:48.682+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 msgr2=0x7f80dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-22T07:03:48.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:48.682+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 msgr2=0x7f80dc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-22T07:03:48.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:48.682+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T07:03:48.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:48.682+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:03:49.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.442+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1293046164 0 0) 0x7f80fc0cd8c0 con 0x7f80dc006200 2024-04-22T07:03:49.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.442+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80e8003520 con 0x7f80dc006200 2024-04-22T07:03:49.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2401962571 0 0) 0x7f80e8003520 con 0x7f80dc006200 2024-04-22T07:03:49.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 msgr2=0x7f80dc005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:03:49.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 --2- 172.21.15.44:0/1301323835 >> v2:172.21.15.70:3300/0 conn(0x7f80dc0036c0 0x7f80dc005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80dc00b000 con 0x7f80dc006200 2024-04-22T07:03:49.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3718881788 0 0) 0x7f80e400f040 con 0x7f80dc006200 2024-04-22T07:03:49.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 >> v1:172.21.15.44:6789/0 conn(0x7f80dc006200 legacy=0x7f80dc0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:49.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80fad9d640 1 -- 172.21.15.44:0/1301323835 reap_dead start 2024-04-22T07:03:49.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80039b0 0x7f80e8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e8004800 0x7f80e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e80052e0 0x7f80e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc0cb4b0 con 0x7f80e80052e0 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f80e800c230 con 0x7f80e80039b0 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f80e800c430 con 0x7f80e8004800 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80039b0 0x7f80e8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80fad9d640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e80052e0 0x7f80e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.443+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e8004800 0x7f80e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.444+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e8004800 msgr2=0x7f80e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.444+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e8004800 0x7f80e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.444+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e80052e0 msgr2=0x7f80e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.444+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e80052e0 0x7f80e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.444+0000 7f80fa59c640 1 -- 172.21.15.44:0/1301323835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e8003520 con 0x7f80e80039b0 2024-04-22T07:03:49.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.445+0000 7f80fa59c640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80039b0 0x7f80e8003db0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0c79e0 tx=0x7f80e8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f697fe4a0f132e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:49.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.445+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4032070 con 0x7f80e80039b0 2024-04-22T07:03:49.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.445+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f402ca10 con 0x7f80e80039b0 2024-04-22T07:03:49.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.445+0000 7f80f959a640 1 -- 172.21.15.44:0/1301323835 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f40051b0 con 0x7f80e80039b0 2024-04-22T07:03:49.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.446+0000 7f8100e55640 1 -- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80039b0 msgr2=0x7f80e8003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:49.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.446+0000 7f8100e55640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80039b0 0x7f80e8003db0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0c79e0 tx=0x7f80e8005830 comp rx=0 tx=0).stop 2024-04-22T07:03:49.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.446+0000 7f8100e55640 1 -- 172.21.15.44:0/1301323835 shutdown_connections 2024-04-22T07:03:49.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.446+0000 7f8100e55640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e8004800 0x7f80e8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.446+0000 7f8100e55640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e80052e0 0x7f80e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.446+0000 7f8100e55640 1 --2- 172.21.15.44:0/1301323835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80039b0 0x7f80e8003db0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.446+0000 7f8100e55640 1 -- 172.21.15.44:0/1301323835 >> 172.21.15.44:0/1301323835 conn(0x7f80fc071f90 msgr2=0x7f80fc0723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:49.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.446+0000 7f8100e55640 1 -- 172.21.15.44:0/1301323835 shutdown_connections 2024-04-22T07:03:49.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.446+0000 7f8100e55640 1 -- 172.21.15.44:0/1301323835 wait complete. 2024-04-22T07:03:49.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.448+0000 7f8100e55640 1 Processor -- start 2024-04-22T07:03:49.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.448+0000 7f8100e55640 1 -- start start 2024-04-22T07:03:49.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.449+0000 7f8100e55640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e80039b0 0x7f80fc1e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.449+0000 7f8100e55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e8004800 0x7f80fc158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.449+0000 7f8100e55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80052e0 0x7f80fc1f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:49.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.449+0000 7f8100e55640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc0cba50 con 0x7f80e8004800 2024-04-22T07:03:49.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.449+0000 7f8100e55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc0cb770 con 0x7f80e80052e0 2024-04-22T07:03:49.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.449+0000 7f8100e55640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f80fc155300 con 0x7f80e80039b0 2024-04-22T07:03:49.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80052e0 0x7f80fc1f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fa59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e80039b0 0x7f80fc1e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80f9d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e8004800 0x7f80fc158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:49.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fa59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e80039b0 0x7f80fc1e1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:51484/0 (socket says 172.21.15.44:51484) 2024-04-22T07:03:49.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80052e0 0x7f80fc1f3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:48348/0 (socket says 172.21.15.44:48348) 2024-04-22T07:03:49.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fa59c640 1 -- 172.21.15.44:0/3918735381 learned_addr learned my addr 172.21.15.44:0/3918735381 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:03:49.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fa59c640 1 -- 172.21.15.44:0/3918735381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80052e0 msgr2=0x7f80fc1f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:49.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fa59c640 1 --2- 172.21.15.44:0/3918735381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80052e0 0x7f80fc1f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fa59c640 1 -- 172.21.15.44:0/3918735381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e8004800 msgr2=0x7f80fc158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fa59c640 1 --2- 172.21.15.44:0/3918735381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e8004800 0x7f80fc158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fa59c640 1 -- 172.21.15.44:0/3918735381 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80fc158060 con 0x7f80e80039b0 2024-04-22T07:03:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80f9d9b640 1 --2- 172.21.15.44:0/3918735381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e8004800 0x7f80fc158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.450+0000 7f80fa59c640 1 --2- 172.21.15.44:0/3918735381 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e80039b0 0x7f80fc1e1d20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80f4038040 tx=0x7f80f4001190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.451+0000 7f80f2ffd640 1 -- 172.21.15.44:0/3918735381 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4045030 con 0x7f80e80039b0 2024-04-22T07:03:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.451+0000 7f80f2ffd640 1 -- 172.21.15.44:0/3918735381 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f4036c90 con 0x7f80e80039b0 2024-04-22T07:03:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.451+0000 7f8100e55640 1 -- 172.21.15.44:0/3918735381 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80fc1e7640 con 0x7f80e80039b0 2024-04-22T07:03:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.451+0000 7f80f2ffd640 1 -- 172.21.15.44:0/3918735381 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f40110b0 con 0x7f80e80039b0 2024-04-22T07:03:49.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.451+0000 7f8100e55640 1 -- 172.21.15.44:0/3918735381 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f80fc1f3550 con 0x7f80e80039b0 2024-04-22T07:03:49.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.451+0000 7f80f2ffd640 1 -- 172.21.15.44:0/3918735381 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f80f4048070 con 0x7f80e80039b0 2024-04-22T07:03:49.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.451+0000 7f80f2ffd640 1 -- 172.21.15.44:0/3918735381 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80f4006b40 con 0x7f80e80039b0 2024-04-22T07:03:49.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.452+0000 7f8100e55640 1 -- 172.21.15.44:0/3918735381 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80fc04fd50 con 0x7f80e80039b0 2024-04-22T07:03:49.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.454+0000 7f80f2ffd640 1 -- 172.21.15.44:0/3918735381 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f80f4006d40 con 0x7f80e80039b0 2024-04-22T07:03:49.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.524+0000 7f8100e55640 1 -- 172.21.15.44:0/3918735381 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f80fc0cc910 con 0x7f80e80039b0 2024-04-22T07:03:49.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.529+0000 7f80f2ffd640 1 -- 172.21.15.44:0/3918735381 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f80f4007240 con 0x7f80e80039b0 2024-04-22T07:03:49.527 INFO:teuthology.orchestra.run.smithi044.stderr:adjusted tunables profile to default 2024-04-22T07:03:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.530+0000 7f8100e55640 1 -- 172.21.15.44:0/3918735381 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e80039b0 msgr2=0x7f80fc1e1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.530+0000 7f8100e55640 1 --2- 172.21.15.44:0/3918735381 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e80039b0 0x7f80fc1e1d20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f80f4038040 tx=0x7f80f4001190 comp rx=0 tx=0).stop 2024-04-22T07:03:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.530+0000 7f8100e55640 1 -- 172.21.15.44:0/3918735381 shutdown_connections 2024-04-22T07:03:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.530+0000 7f8100e55640 1 --2- 172.21.15.44:0/3918735381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80e80052e0 0x7f80fc1f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.530+0000 7f8100e55640 1 --2- 172.21.15.44:0/3918735381 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f80e8004800 0x7f80fc158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.530+0000 7f8100e55640 1 --2- 172.21.15.44:0/3918735381 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f80e80039b0 0x7f80fc1e1d20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.530+0000 7f8100e55640 1 -- 172.21.15.44:0/3918735381 >> 172.21.15.44:0/3918735381 conn(0x7f80fc071f90 msgr2=0x7f80fc15a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.530+0000 7f8100e55640 1 -- 172.21.15.44:0/3918735381 shutdown_connections 2024-04-22T07:03:49.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:49.530+0000 7f8100e55640 1 -- 172.21.15.44:0/3918735381 wait complete. 2024-04-22T07:03:49.539 INFO:tasks.ceph:check_enable_crimson: False 2024-04-22T07:03:49.539 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-22T07:03:49.540 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:49.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-22T07:03:49.565 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:49.565 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-22T07:03:49.568 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:49.570+0000 7f74af443040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:03:49.592 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:49.592 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-22T07:03:49.657 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:03:49.657 DEBUG:teuthology.orchestra.run.smithi044:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-22T07:03:49.722 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:49.722 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-22T07:03:49.731 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:49.734+0000 7f74af443040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:03:49.751 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:49.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-22T07:03:49.787 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:49.789+0000 7f74af443040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:03:49.818 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:49.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-22T07:03:49.884 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-04-22T07:03:49.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-22T07:03:49.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new b02b7a83-3c8d-4357-b2cb-4d0a5868a452 0 2024-04-22T07:03:50.047 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:50.049+0000 7f74af443040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:03:50.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.093+0000 7f91d47bd640 1 Processor -- start 2024-04-22T07:03:50.095 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.093+0000 7f91d47bd640 1 -- start start 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.094+0000 7f91d47bd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc0c4070 0x7f91cc0cba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.094+0000 7f91d47bd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91cc0c3d90 con 0x7f91cc15e8a0 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.094+0000 7f91d47bd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91cc15e3a0 con 0x7f91cc154f80 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.094+0000 7f91d47bd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc0b03f0 con 0x7f91cc0c4070 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.094+0000 7f91d1d58640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc0c4070 0x7f91cc0cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.095+0000 7f91d2559640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f91cc154f80 0x7f91cc0ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59218/0 (socket says 172.21.15.70:59218) 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.095+0000 7f91d2559640 1 -- 172.21.15.70:0/1841960393 learned_addr learned my addr 172.21.15.70:0/1841960393 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.095+0000 7f91d1557640 1 -- 172.21.15.70:0/1841960393 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3605663683 0 0) 0x7f91cc15e3a0 con 0x7f91cc154f80 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.095+0000 7f91d1557640 1 -- 172.21.15.70:0/1841960393 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91b8003520 con 0x7f91cc154f80 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.095+0000 7f91d1d58640 1 -- 172.21.15.70:0/1841960393 >> v1:172.21.15.70:6789/0 conn(0x7f91cc154f80 legacy=0x7f91cc0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.095+0000 7f91d1d58640 1 -- 172.21.15.70:0/1841960393 >> v1:172.21.15.44:6789/0 conn(0x7f91cc15e8a0 legacy=0x7f91cc165730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.095+0000 7f91d1d58640 1 -- 172.21.15.70:0/1841960393 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc0b06d0 con 0x7f91cc0c4070 2024-04-22T07:03:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d1d58640 1 --2- 172.21.15.70:0/1841960393 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc0c4070 0x7f91cc0cba10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0097d0 tx=0x7f91bc02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d05d6d7e8a0cc849 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d1557640 1 -- 172.21.15.70:0/1841960393 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc034070 con 0x7f91cc0c4070 2024-04-22T07:03:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d1557640 1 -- 172.21.15.70:0/1841960393 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc030880 con 0x7f91cc0c4070 2024-04-22T07:03:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d1557640 1 -- 172.21.15.70:0/1841960393 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc030c50 con 0x7f91cc0c4070 2024-04-22T07:03:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d47bd640 1 -- 172.21.15.70:0/1841960393 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc0c4070 msgr2=0x7f91cc0cba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d47bd640 1 --2- 172.21.15.70:0/1841960393 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc0c4070 0x7f91cc0cba10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0097d0 tx=0x7f91bc02b9c0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d47bd640 1 -- 172.21.15.70:0/1841960393 shutdown_connections 2024-04-22T07:03:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d47bd640 1 --2- 172.21.15.70:0/1841960393 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc0c4070 0x7f91cc0cba10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d47bd640 1 -- 172.21.15.70:0/1841960393 >> 172.21.15.70:0/1841960393 conn(0x7f91cc070160 msgr2=0x7f91cc0705f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d47bd640 1 -- 172.21.15.70:0/1841960393 shutdown_connections 2024-04-22T07:03:50.098 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.096+0000 7f91d47bd640 1 -- 172.21.15.70:0/1841960393 wait complete. 2024-04-22T07:03:50.098 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:50.097+0000 7f74af443040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:03:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.097+0000 7f91d47bd640 1 Processor -- start 2024-04-22T07:03:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d47bd640 1 -- start start 2024-04-22T07:03:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d47bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc0c4070 0x7f91cc1e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d47bd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc154f80 0x7f91cc06ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d47bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91cc15e8a0 0x7f91cc06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d47bd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0c3a50 con 0x7f91cc15e8a0 2024-04-22T07:03:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d47bd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc0b00f0 con 0x7f91cc0c4070 2024-04-22T07:03:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d47bd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f91cc155360 con 0x7f91cc154f80 2024-04-22T07:03:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d2559640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc0c4070 0x7f91cc1e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d1d58640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc154f80 0x7f91cc06ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d2559640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc0c4070 0x7f91cc1e71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48534/0 (socket says 172.21.15.70:48534) 2024-04-22T07:03:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d2559640 1 -- 172.21.15.70:0/2005787902 learned_addr learned my addr 172.21.15.70:0/2005787902 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.098+0000 7f91d2d5a640 1 --2- 172.21.15.70:0/2005787902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91cc15e8a0 0x7f91cc06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91d1d58640 1 -- 172.21.15.70:0/2005787902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc0c4070 msgr2=0x7f91cc1e71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91d1d58640 1 --2- 172.21.15.70:0/2005787902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc0c4070 0x7f91cc1e71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91d1d58640 1 -- 172.21.15.70:0/2005787902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91cc15e8a0 msgr2=0x7f91cc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91d1d58640 1 --2- 172.21.15.70:0/2005787902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91cc15e8a0 0x7f91cc06d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91d1d58640 1 -- 172.21.15.70:0/2005787902 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc1f1010 con 0x7f91cc154f80 2024-04-22T07:03:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91d2559640 1 --2- 172.21.15.70:0/2005787902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc0c4070 0x7f91cc1e71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91d1d58640 1 --2- 172.21.15.70:0/2005787902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc154f80 0x7f91cc06ccb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91bc033000 tx=0x7f91bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91c2ffd640 1 -- 172.21.15.70:0/2005787902 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc047070 con 0x7f91cc154f80 2024-04-22T07:03:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91c2ffd640 1 -- 172.21.15.70:0/2005787902 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc030680 con 0x7f91cc154f80 2024-04-22T07:03:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91d47bd640 1 -- 172.21.15.70:0/2005787902 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91cc1f12f0 con 0x7f91cc154f80 2024-04-22T07:03:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91c2ffd640 1 -- 172.21.15.70:0/2005787902 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc034040 con 0x7f91cc154f80 2024-04-22T07:03:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91d47bd640 1 -- 172.21.15.70:0/2005787902 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91cc1f18b0 con 0x7f91cc154f80 2024-04-22T07:03:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91c2ffd640 1 -- 172.21.15.70:0/2005787902 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f91bc04a070 con 0x7f91cc154f80 2024-04-22T07:03:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.099+0000 7f91c2ffd640 1 -- 172.21.15.70:0/2005787902 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f91bc04c780 con 0x7f91cc154f80 2024-04-22T07:03:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.100+0000 7f91d47bd640 1 -- 172.21.15.70:0/2005787902 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91cc0c5680 con 0x7f91cc154f80 2024-04-22T07:03:50.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.102+0000 7f91c2ffd640 1 -- 172.21.15.70:0/2005787902 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f91cc0c5680 con 0x7f91cc154f80 2024-04-22T07:03:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.171+0000 7f91d47bd640 1 -- 172.21.15.70:0/2005787902 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b02b7a83-3c8d-4357-b2cb-4d0a5868a452", "id": 0} v 0) -- 0x7f91cc069210 con 0x7f91cc154f80 2024-04-22T07:03:50.172 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.174+0000 7f91c2ffd640 1 -- 172.21.15.70:0/2005787902 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b02b7a83-3c8d-4357-b2cb-4d0a5868a452", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f91bc051070 con 0x7f91cc154f80 2024-04-22T07:03:50.173 INFO:teuthology.orchestra.run.smithi070.stdout:0 2024-04-22T07:03:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.175+0000 7f91d47bd640 1 -- 172.21.15.70:0/2005787902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc154f80 msgr2=0x7f91cc06ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.175+0000 7f91d47bd640 1 --2- 172.21.15.70:0/2005787902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc154f80 0x7f91cc06ccb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91bc033000 tx=0x7f91bc0046c0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.175+0000 7f91d47bd640 1 -- 172.21.15.70:0/2005787902 shutdown_connections 2024-04-22T07:03:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.175+0000 7f91d47bd640 1 --2- 172.21.15.70:0/2005787902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f91cc154f80 0x7f91cc06ccb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.175+0000 7f91d47bd640 1 --2- 172.21.15.70:0/2005787902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f91cc15e8a0 0x7f91cc06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.175+0000 7f91d47bd640 1 --2- 172.21.15.70:0/2005787902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91cc0c4070 0x7f91cc1e71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.175+0000 7f91d47bd640 1 -- 172.21.15.70:0/2005787902 >> 172.21.15.70:0/2005787902 conn(0x7f91cc070160 msgr2=0x7f91cc152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.175+0000 7f91d47bd640 1 -- 172.21.15.70:0/2005787902 shutdown_connections 2024-04-22T07:03:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.175+0000 7f91d47bd640 1 -- 172.21.15.70:0/2005787902 wait complete. 2024-04-22T07:03:50.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new d59d97eb-4bd1-4884-bd1a-a028ee23283f 1 2024-04-22T07:03:50.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.283+0000 7febe6bf4640 1 Processor -- start 2024-04-22T07:03:50.282 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.283+0000 7febe6bf4640 1 -- start start 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe6bf4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe0071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe6bf4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7febe00c7350 0x7febe00c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe6bf4640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe0158c70 con 0x7febe0152a80 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe6bf4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7febe0159560 con 0x7febe00c7350 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe6bf4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7febe01599f0 con 0x7febe0073d60 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe5191640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7febe00c7350 0x7febe00c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febdffff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe0071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe5191640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7febe00c7350 0x7febe00c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48540/0 (socket says 172.21.15.70:48540) 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe5191640 1 -- 172.21.15.70:0/2819677893 learned_addr learned my addr 172.21.15.70:0/2819677893 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe5191640 1 -- 172.21.15.70:0/2819677893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 msgr2=0x7febe0071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe5191640 1 --2- 172.21.15.70:0/2819677893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe0071ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe5191640 1 -- 172.21.15.70:0/2819677893 >> v1:172.21.15.44:6789/0 conn(0x7febe0152a80 legacy=0x7febe00c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febe5191640 1 -- 172.21.15.70:0/2819677893 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe00727e0 con 0x7febe00c7350 2024-04-22T07:03:50.285 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.284+0000 7febdffff640 1 --2- 172.21.15.70:0/2819677893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe0071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.285+0000 7febe5191640 1 --2- 172.21.15.70:0/2819677893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febe00c7350 0x7febe00c6060 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7febd8009e50 tx=0x7febd802bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=219b2a1f1d68afbb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.285+0000 7febdf7fe640 1 -- 172.21.15.70:0/2819677893 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd802cb90 con 0x7febe00c7350 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.285+0000 7febdf7fe640 1 -- 172.21.15.70:0/2819677893 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd802cd30 con 0x7febe00c7350 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.285+0000 7febdf7fe640 1 -- 172.21.15.70:0/2819677893 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd8034610 con 0x7febe00c7350 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.286+0000 7febe6bf4640 1 -- 172.21.15.70:0/2819677893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febe00c7350 msgr2=0x7febe00c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.286+0000 7febe6bf4640 1 --2- 172.21.15.70:0/2819677893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febe00c7350 0x7febe00c6060 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7febd8009e50 tx=0x7febd802bff0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.286+0000 7febe6bf4640 1 -- 172.21.15.70:0/2819677893 shutdown_connections 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.286+0000 7febe6bf4640 1 --2- 172.21.15.70:0/2819677893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febe00c7350 0x7febe00c6060 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.286+0000 7febe6bf4640 1 --2- 172.21.15.70:0/2819677893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe0071ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.286+0000 7febe6bf4640 1 -- 172.21.15.70:0/2819677893 >> 172.21.15.70:0/2819677893 conn(0x7febe00cc9d0 msgr2=0x7febe00ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.286+0000 7febe6bf4640 1 -- 172.21.15.70:0/2819677893 shutdown_connections 2024-04-22T07:03:50.286 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.286+0000 7febe6bf4640 1 -- 172.21.15.70:0/2819677893 wait complete. 2024-04-22T07:03:50.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.287+0000 7febe6bf4640 1 Processor -- start 2024-04-22T07:03:50.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.287+0000 7febe6bf4640 1 -- start start 2024-04-22T07:03:50.289 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe6bf4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe014d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe6bf4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febe00c7350 0x7febe0149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe6bf4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febe0152a80 0x7febe01f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe6bf4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7febe01591f0 con 0x7febe0152a80 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe6bf4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7febe0158f10 con 0x7febe00c7350 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe6bf4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7febe00c8980 con 0x7febe0073d60 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe4990640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe014d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febdffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febe00c7350 0x7febe0149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe4990640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe014d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:42218/0 (socket says 172.21.15.70:42218) 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe4990640 1 -- 172.21.15.70:0/1057415791 learned_addr learned my addr 172.21.15.70:0/1057415791 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe5191640 1 --2- 172.21.15.70:0/1057415791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febe0152a80 0x7febe01f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe4990640 1 -- 172.21.15.70:0/1057415791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febe00c7350 msgr2=0x7febe0149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe4990640 1 --2- 172.21.15.70:0/1057415791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febe00c7350 0x7febe0149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe4990640 1 -- 172.21.15.70:0/1057415791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febe0152a80 msgr2=0x7febe01f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe4990640 1 --2- 172.21.15.70:0/1057415791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febe0152a80 0x7febe01f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe4990640 1 -- 172.21.15.70:0/1057415791 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe014dee0 con 0x7febe0073d60 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febdffff640 1 --2- 172.21.15.70:0/1057415791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febe00c7350 0x7febe0149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.288+0000 7febe4990640 1 --2- 172.21.15.70:0/1057415791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe014d9a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7febd400c340 tx=0x7febd400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.289+0000 7febdd7fa640 1 -- 172.21.15.70:0/1057415791 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4017070 con 0x7febe0073d60 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.289+0000 7febdd7fa640 1 -- 172.21.15.70:0/1057415791 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd40052d0 con 0x7febe0073d60 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.289+0000 7febe6bf4640 1 -- 172.21.15.70:0/1057415791 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe00c3e50 con 0x7febe0073d60 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.289+0000 7febdd7fa640 1 -- 172.21.15.70:0/1057415791 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4003850 con 0x7febe0073d60 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.289+0000 7febe6bf4640 1 -- 172.21.15.70:0/1057415791 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7febe00c43f0 con 0x7febe0073d60 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.289+0000 7febdd7fa640 1 -- 172.21.15.70:0/1057415791 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7febd401a070 con 0x7febe0073d60 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.289+0000 7febdd7fa640 1 -- 172.21.15.70:0/1057415791 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7febd4011bc0 con 0x7febe0073d60 2024-04-22T07:03:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.289+0000 7febe6bf4640 1 -- 172.21.15.70:0/1057415791 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe0069210 con 0x7febe0073d60 2024-04-22T07:03:50.292 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.292+0000 7febdd7fa640 1 -- 172.21.15.70:0/1057415791 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7febd4021070 con 0x7febe0073d60 2024-04-22T07:03:50.354 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:50.355+0000 7f2de9133040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T07:03:50.362 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.364+0000 7febe6bf4640 1 -- 172.21.15.70:0/1057415791 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d59d97eb-4bd1-4884-bd1a-a028ee23283f", "id": 1} v 0) -- 0x7febe014e160 con 0x7febe0073d60 2024-04-22T07:03:50.367 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.369+0000 7febdd7fa640 1 -- 172.21.15.70:0/1057415791 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d59d97eb-4bd1-4884-bd1a-a028ee23283f", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7febd4011dc0 con 0x7febe0073d60 2024-04-22T07:03:50.368 INFO:teuthology.orchestra.run.smithi070.stdout:1 2024-04-22T07:03:50.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.370+0000 7febe6bf4640 1 -- 172.21.15.70:0/1057415791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 msgr2=0x7febe014d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.370+0000 7febe6bf4640 1 --2- 172.21.15.70:0/1057415791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe014d9a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7febd400c340 tx=0x7febd400c800 comp rx=0 tx=0).stop 2024-04-22T07:03:50.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.370+0000 7febe6bf4640 1 -- 172.21.15.70:0/1057415791 shutdown_connections 2024-04-22T07:03:50.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.370+0000 7febe6bf4640 1 --2- 172.21.15.70:0/1057415791 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7febe0152a80 0x7febe01f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.370+0000 7febe6bf4640 1 --2- 172.21.15.70:0/1057415791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febe00c7350 0x7febe0149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.370+0000 7febe6bf4640 1 --2- 172.21.15.70:0/1057415791 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7febe0073d60 0x7febe014d9a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.370+0000 7febe6bf4640 1 -- 172.21.15.70:0/1057415791 >> 172.21.15.70:0/1057415791 conn(0x7febe00cc9d0 msgr2=0x7febe01506b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:50.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.370+0000 7febe6bf4640 1 -- 172.21.15.70:0/1057415791 shutdown_connections 2024-04-22T07:03:50.369 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.370+0000 7febe6bf4640 1 -- 172.21.15.70:0/1057415791 wait complete. 2024-04-22T07:03:50.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new df8331b0-946c-4b09-8cab-af825d9a1344 2 2024-04-22T07:03:50.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.479+0000 7f7f43f96640 1 Processor -- start 2024-04-22T07:03:50.477 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.479+0000 7f7f43f96640 1 -- start start 2024-04-22T07:03:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f43f96640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f3c0c4400 0x7f7f3c1632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f43f96640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f3c0c9220 con 0x7f7f3c06d090 2024-04-22T07:03:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f43f96640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f3c0c8f80 con 0x7f7f3c151c40 2024-04-22T07:03:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f43f96640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f7f3c0c4ac0 con 0x7f7f3c0c4400 2024-04-22T07:03:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f42533640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f3c0c4400 0x7f7f3c1632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f41d32640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7f3c151c40 0x7f7f3c157a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59226/0 (socket says 172.21.15.70:59226) 2024-04-22T07:03:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f42533640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f3c0c4400 0x7f7f3c1632f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48556/0 (socket says 172.21.15.70:48556) 2024-04-22T07:03:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f42533640 1 -- 172.21.15.70:0/3349229483 learned_addr learned my addr 172.21.15.70:0/3349229483 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f42533640 1 -- 172.21.15.70:0/3349229483 >> v1:172.21.15.70:6789/0 conn(0x7f7f3c151c40 legacy=0x7f7f3c157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f42533640 1 -- 172.21.15.70:0/3349229483 >> v1:172.21.15.44:6789/0 conn(0x7f7f3c06d090 legacy=0x7f7f3c0c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f40d30640 1 -- 172.21.15.70:0/3349229483 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006785155 0 0) 0x7f7f3c0c8f80 con 0x7f7f3c151c40 2024-04-22T07:03:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.480+0000 7f7f42533640 1 -- 172.21.15.70:0/3349229483 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c0c9d90 con 0x7f7f3c0c4400 2024-04-22T07:03:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.481+0000 7f7f42533640 1 --2- 172.21.15.70:0/3349229483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f3c0c4400 0x7f7f3c1632f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009e50 tx=0x7f7f3402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e063e866fcd1596e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.481+0000 7f7f40d30640 1 -- 172.21.15.70:0/3349229483 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3402c960 con 0x7f7f3c0c4400 2024-04-22T07:03:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.481+0000 7f7f40d30640 1 -- 172.21.15.70:0/3349229483 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3402cb00 con 0x7f7f3c0c4400 2024-04-22T07:03:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.481+0000 7f7f40d30640 1 -- 172.21.15.70:0/3349229483 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340343f0 con 0x7f7f3c0c4400 2024-04-22T07:03:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.482+0000 7f7f43f96640 1 -- 172.21.15.70:0/3349229483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f3c0c4400 msgr2=0x7f7f3c1632f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.482+0000 7f7f43f96640 1 --2- 172.21.15.70:0/3349229483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f3c0c4400 0x7f7f3c1632f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009e50 tx=0x7f7f3402be40 comp rx=0 tx=0).stop 2024-04-22T07:03:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.482+0000 7f7f43f96640 1 -- 172.21.15.70:0/3349229483 shutdown_connections 2024-04-22T07:03:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.482+0000 7f7f43f96640 1 --2- 172.21.15.70:0/3349229483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f3c0c4400 0x7f7f3c1632f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.482+0000 7f7f43f96640 1 -- 172.21.15.70:0/3349229483 >> 172.21.15.70:0/3349229483 conn(0x7f7f3c0c9f90 msgr2=0x7f7f3c0ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.482+0000 7f7f43f96640 1 -- 172.21.15.70:0/3349229483 shutdown_connections 2024-04-22T07:03:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.482+0000 7f7f43f96640 1 -- 172.21.15.70:0/3349229483 wait complete. 2024-04-22T07:03:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.483+0000 7f7f43f96640 1 Processor -- start 2024-04-22T07:03:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.483+0000 7f7f43f96640 1 -- start start 2024-04-22T07:03:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f43f96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f3c06d090 0x7f7f3c14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f43f96640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0c4400 0x7f7f3c155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f43f96640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f3c151c40 0x7f7f3c0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f43f96640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c0c9870 con 0x7f7f3c0c4400 2024-04-22T07:03:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f43f96640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3c0c5040 con 0x7f7f3c06d090 2024-04-22T07:03:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f43f96640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3c0c9570 con 0x7f7f3c151c40 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f41d32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f3c06d090 0x7f7f3c14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f41d32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f3c06d090 0x7f7f3c14fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48560/0 (socket says 172.21.15.70:48560) 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f41d32640 1 -- 172.21.15.70:0/1695707642 learned_addr learned my addr 172.21.15.70:0/1695707642 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f42533640 1 --2- 172.21.15.70:0/1695707642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f3c151c40 0x7f7f3c0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f41531640 1 --2- 172.21.15.70:0/1695707642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0c4400 0x7f7f3c155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f42533640 1 -- 172.21.15.70:0/1695707642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f3c06d090 msgr2=0x7f7f3c14fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f42533640 1 --2- 172.21.15.70:0/1695707642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f3c06d090 0x7f7f3c14fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f42533640 1 -- 172.21.15.70:0/1695707642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0c4400 msgr2=0x7f7f3c155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f42533640 1 --2- 172.21.15.70:0/1695707642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0c4400 0x7f7f3c155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f42533640 1 -- 172.21.15.70:0/1695707642 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3c1f02f0 con 0x7f7f3c151c40 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f42533640 1 --2- 172.21.15.70:0/1695707642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f3c151c40 0x7f7f3c0cc920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7f34032000 tx=0x7f7f34004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f327fc640 1 -- 172.21.15.70:0/1695707642 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34043020 con 0x7f7f3c151c40 2024-04-22T07:03:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f327fc640 1 -- 172.21.15.70:0/1695707642 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3403e450 con 0x7f7f3c151c40 2024-04-22T07:03:50.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.484+0000 7f7f327fc640 1 -- 172.21.15.70:0/1695707642 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34009510 con 0x7f7f3c151c40 2024-04-22T07:03:50.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.485+0000 7f7f43f96640 1 -- 172.21.15.70:0/1695707642 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3c14f230 con 0x7f7f3c151c40 2024-04-22T07:03:50.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.485+0000 7f7f43f96640 1 -- 172.21.15.70:0/1695707642 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f3c1f0570 con 0x7f7f3c151c40 2024-04-22T07:03:50.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.485+0000 7f7f327fc640 1 -- 172.21.15.70:0/1695707642 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7f34046020 con 0x7f7f3c151c40 2024-04-22T07:03:50.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.485+0000 7f7f327fc640 1 -- 172.21.15.70:0/1695707642 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f7f34048870 con 0x7f7f3c151c40 2024-04-22T07:03:50.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.486+0000 7f7f43f96640 1 -- 172.21.15.70:0/1695707642 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f3c069210 con 0x7f7f3c151c40 2024-04-22T07:03:50.487 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.488+0000 7f7f327fc640 1 -- 172.21.15.70:0/1695707642 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7f3404d030 con 0x7f7f3c151c40 2024-04-22T07:03:50.506 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:50.508+0000 7f2de9133040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:03:50.559 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.560+0000 7f7f43f96640 1 -- 172.21.15.70:0/1695707642 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "df8331b0-946c-4b09-8cab-af825d9a1344", "id": 2} v 0) -- 0x7f7f3c070f10 con 0x7f7f3c151c40 2024-04-22T07:03:50.561 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:50.563+0000 7f2de9133040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T07:03:50.564 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.566+0000 7f7f327fc640 1 -- 172.21.15.70:0/1695707642 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df8331b0-946c-4b09-8cab-af825d9a1344", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f7f3c1f02f0 con 0x7f7f3c151c40 2024-04-22T07:03:50.564 INFO:teuthology.orchestra.run.smithi070.stdout:2 2024-04-22T07:03:50.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.566+0000 7f7f43f96640 1 -- 172.21.15.70:0/1695707642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f3c151c40 msgr2=0x7f7f3c0cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.566+0000 7f7f43f96640 1 --2- 172.21.15.70:0/1695707642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f3c151c40 0x7f7f3c0cc920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7f34032000 tx=0x7f7f34004a40 comp rx=0 tx=0).stop 2024-04-22T07:03:50.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.566+0000 7f7f43f96640 1 -- 172.21.15.70:0/1695707642 shutdown_connections 2024-04-22T07:03:50.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.566+0000 7f7f43f96640 1 --2- 172.21.15.70:0/1695707642 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7f3c151c40 0x7f7f3c0cc920 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.565 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.566+0000 7f7f43f96640 1 --2- 172.21.15.70:0/1695707642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7f3c0c4400 0x7f7f3c155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.567+0000 7f7f43f96640 1 --2- 172.21.15.70:0/1695707642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f3c06d090 0x7f7f3c14fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.567+0000 7f7f43f96640 1 -- 172.21.15.70:0/1695707642 >> 172.21.15.70:0/1695707642 conn(0x7f7f3c0c9f90 msgr2=0x7f7f3c06fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:50.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.567+0000 7f7f43f96640 1 -- 172.21.15.70:0/1695707642 shutdown_connections 2024-04-22T07:03:50.566 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.567+0000 7f7f43f96640 1 -- 172.21.15.70:0/1695707642 wait complete. 2024-04-22T07:03:50.575 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new a2849b7d-4d44-4107-a97f-f24c6b94ee4f 3 2024-04-22T07:03:50.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.674+0000 7f88a924a640 1 Processor -- start 2024-04-22T07:03:50.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.674+0000 7f88a924a640 1 -- start start 2024-04-22T07:03:50.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a924a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88a4067d00 0x7f88a40c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a924a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a40c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a924a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88a40c8360 con 0x7f88a4067920 2024-04-22T07:03:50.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a924a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f88a406bc10 con 0x7f88a4067d00 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a924a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f88a415e550 con 0x7f88a4070df0 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88a4067d00 0x7f88a40c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88a4067d00 0x7f88a40c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48562/0 (socket says 172.21.15.70:48562) 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a259c640 1 -- 172.21.15.70:0/1090537537 learned_addr learned my addr 172.21.15.70:0/1090537537 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a359e640 1 --2- 172.21.15.70:0/1090537537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a40c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a359e640 1 -- 172.21.15.70:0/1090537537 >> v2:172.21.15.70:3300/0 conn(0x7f88a4067d00 msgr2=0x7f88a40c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a359e640 1 --2- 172.21.15.70:0/1090537537 >> v2:172.21.15.70:3300/0 conn(0x7f88a4067d00 0x7f88a40c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a359e640 1 -- 172.21.15.70:0/1090537537 >> v1:172.21.15.70:6789/0 conn(0x7f88a4067920 legacy=0x7f88a415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a1d9b640 1 -- 172.21.15.70:0/1090537537 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990467496 0 0) 0x7f88a40c8360 con 0x7f88a4067920 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a359e640 1 -- 172.21.15.70:0/1090537537 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a415e830 con 0x7f88a4070df0 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.675+0000 7f88a259c640 1 --2- 172.21.15.70:0/1090537537 >> v2:172.21.15.70:3300/0 conn(0x7f88a4067d00 0x7f88a40c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.676+0000 7f88a359e640 1 --2- 172.21.15.70:0/1090537537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a40c8d50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f889c009dc0 tx=0x7f889c02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9789f113e43f52c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.676+0000 7f88a1d9b640 1 -- 172.21.15.70:0/1090537537 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c039070 con 0x7f88a4070df0 2024-04-22T07:03:50.675 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.676+0000 7f88a1d9b640 1 -- 172.21.15.70:0/1090537537 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889c02ccd0 con 0x7f88a4070df0 2024-04-22T07:03:50.676 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.676+0000 7f88a1d9b640 1 -- 172.21.15.70:0/1090537537 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c034660 con 0x7f88a4070df0 2024-04-22T07:03:50.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.676+0000 7f88a924a640 1 -- 172.21.15.70:0/1090537537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 msgr2=0x7f88a40c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.676+0000 7f88a924a640 1 --2- 172.21.15.70:0/1090537537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a40c8d50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f889c009dc0 tx=0x7f889c02bf20 comp rx=0 tx=0).stop 2024-04-22T07:03:50.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.677+0000 7f88a924a640 1 -- 172.21.15.70:0/1090537537 shutdown_connections 2024-04-22T07:03:50.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.677+0000 7f88a924a640 1 --2- 172.21.15.70:0/1090537537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a40c8d50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.677+0000 7f88a924a640 1 --2- 172.21.15.70:0/1090537537 >> v2:172.21.15.70:3300/0 conn(0x7f88a4067d00 0x7f88a40c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.679 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.677+0000 7f88a924a640 1 -- 172.21.15.70:0/1090537537 >> 172.21.15.70:0/1090537537 conn(0x7f88a40c9f90 msgr2=0x7f88a40ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.677+0000 7f88a924a640 1 -- 172.21.15.70:0/1090537537 shutdown_connections 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.677+0000 7f88a924a640 1 -- 172.21.15.70:0/1090537537 wait complete. 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.678+0000 7f88a924a640 1 Processor -- start 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.678+0000 7f88a924a640 1 -- start start 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a924a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a4067920 0x7f88a41f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a924a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88a4067d00 0x7f88a41533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a924a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a4157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a924a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f88a415e230 con 0x7f88a4067d00 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a924a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f88a415edf0 con 0x7f88a4067920 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a924a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f88a415f0d0 con 0x7f88a4070df0 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a4067920 0x7f88a41f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a359e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a4157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a4067920 0x7f88a41f45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48564/0 (socket says 172.21.15.70:48564) 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a2d9d640 1 -- 172.21.15.70:0/3900317369 learned_addr learned my addr 172.21.15.70:0/3900317369 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a359e640 1 --2- 172.21.15.70:0/3900317369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a4157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:42256/0 (socket says 172.21.15.70:42256) 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a259c640 1 --2- 172.21.15.70:0/3900317369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88a4067d00 0x7f88a41533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a359e640 1 -- 172.21.15.70:0/3900317369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a4067920 msgr2=0x7f88a41f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a359e640 1 --2- 172.21.15.70:0/3900317369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a4067920 0x7f88a41f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a359e640 1 -- 172.21.15.70:0/3900317369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88a4067d00 msgr2=0x7f88a41533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a359e640 1 --2- 172.21.15.70:0/3900317369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88a4067d00 0x7f88a41533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a359e640 1 -- 172.21.15.70:0/3900317369 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a41551e0 con 0x7f88a4070df0 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a259c640 1 --2- 172.21.15.70:0/3900317369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88a4067d00 0x7f88a41533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a359e640 1 --2- 172.21.15.70:0/3900317369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a4157120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f889c033000 tx=0x7f889c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f888b7fe640 1 -- 172.21.15.70:0/3900317369 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c039050 con 0x7f88a4070df0 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f888b7fe640 1 -- 172.21.15.70:0/3900317369 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889c03a040 con 0x7f88a4070df0 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f88a924a640 1 -- 172.21.15.70:0/3900317369 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a41554c0 con 0x7f88a4070df0 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.679+0000 7f888b7fe640 1 -- 172.21.15.70:0/3900317369 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c009450 con 0x7f88a4070df0 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.680+0000 7f88a924a640 1 -- 172.21.15.70:0/3900317369 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a4155a80 con 0x7f88a4070df0 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.680+0000 7f888b7fe640 1 -- 172.21.15.70:0/3900317369 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f889c047020 con 0x7f88a4070df0 2024-04-22T07:03:50.681 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.680+0000 7f888b7fe640 1 -- 172.21.15.70:0/3900317369 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f889c0409d0 con 0x7f88a4070df0 2024-04-22T07:03:50.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.680+0000 7f88a924a640 1 -- 172.21.15.70:0/3900317369 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a4069210 con 0x7f88a4070df0 2024-04-22T07:03:50.682 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.682+0000 7f888b7fe640 1 -- 172.21.15.70:0/3900317369 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f889c040c70 con 0x7f88a4070df0 2024-04-22T07:03:50.754 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.756+0000 7f88a924a640 1 -- 172.21.15.70:0/3900317369 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a2849b7d-4d44-4107-a97f-f24c6b94ee4f", "id": 3} v 0) -- 0x7f88a40685a0 con 0x7f88a4070df0 2024-04-22T07:03:50.760 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.761+0000 7f888b7fe640 1 -- 172.21.15.70:0/3900317369 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a2849b7d-4d44-4107-a97f-f24c6b94ee4f", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f889c040540 con 0x7f88a4070df0 2024-04-22T07:03:50.760 INFO:teuthology.orchestra.run.smithi070.stdout:3 2024-04-22T07:03:50.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.762+0000 7f88a924a640 1 -- 172.21.15.70:0/3900317369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 msgr2=0x7f88a4157120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.762+0000 7f88a924a640 1 --2- 172.21.15.70:0/3900317369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a4157120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f889c033000 tx=0x7f889c004a40 comp rx=0 tx=0).stop 2024-04-22T07:03:50.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.762+0000 7f88a924a640 1 -- 172.21.15.70:0/3900317369 shutdown_connections 2024-04-22T07:03:50.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.762+0000 7f88a924a640 1 --2- 172.21.15.70:0/3900317369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f88a4070df0 0x7f88a4157120 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.762+0000 7f88a924a640 1 --2- 172.21.15.70:0/3900317369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f88a4067d00 0x7f88a41533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.762+0000 7f88a924a640 1 --2- 172.21.15.70:0/3900317369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a4067920 0x7f88a41f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.762+0000 7f88a924a640 1 -- 172.21.15.70:0/3900317369 >> 172.21.15.70:0/3900317369 conn(0x7f88a40c9f90 msgr2=0x7f88a40731a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:50.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.762+0000 7f88a924a640 1 -- 172.21.15.70:0/3900317369 shutdown_connections 2024-04-22T07:03:50.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.762+0000 7f88a924a640 1 -- 172.21.15.70:0/3900317369 wait complete. 2024-04-22T07:03:50.771 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new e6d3626b-9227-40fc-af5d-ff49b5da155f 4 2024-04-22T07:03:50.814 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:50.815+0000 7f2de9133040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T07:03:50.827 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:50.830+0000 7f74af443040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:03:50.861 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:50.863+0000 7f2de9133040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T07:03:50.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.869+0000 7f6b89e77640 1 Processor -- start 2024-04-22T07:03:50.868 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.869+0000 7f6b89e77640 1 -- start start 2024-04-22T07:03:50.869 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.870+0000 7f6b89e77640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6b84152070 0x7f6b8415b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.870+0000 7f6b89e77640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b840c4c30 con 0x7f6b84151c90 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.870+0000 7f6b89e77640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b840c4ef0 con 0x7f6b8406cd30 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.870+0000 7f6b89e77640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f6b84072ab0 con 0x7f6b84152070 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.870+0000 7f6b83fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6b8406cd30 0x7f6b841623e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59242/0 (socket says 172.21.15.70:59242) 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.870+0000 7f6b83fff640 1 -- 172.21.15.70:0/721661263 learned_addr learned my addr 172.21.15.70:0/721661263 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.870+0000 7f6b7bfff640 1 --2- 172.21.15.70:0/721661263 >> v2:172.21.15.44:3300/0 conn(0x7f6b84152070 0x7f6b8415b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.871+0000 7f6b82ffd640 1 -- 172.21.15.70:0/721661263 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386944058 0 0) 0x7f6b840c4ef0 con 0x7f6b8406cd30 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.871+0000 7f6b82ffd640 1 -- 172.21.15.70:0/721661263 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b70003520 con 0x7f6b8406cd30 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.871+0000 7f6b82ffd640 1 -- 172.21.15.70:0/721661263 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 177404002 0 0) 0x7f6b840c4c30 con 0x7f6b84151c90 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.871+0000 7f6b82ffd640 1 -- 172.21.15.70:0/721661263 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b840c4ef0 con 0x7f6b84151c90 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.871+0000 7f6b7bfff640 1 -- 172.21.15.70:0/721661263 >> v1:172.21.15.70:6789/0 conn(0x7f6b8406cd30 legacy=0x7f6b841623e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.871+0000 7f6b7bfff640 1 -- 172.21.15.70:0/721661263 >> v1:172.21.15.44:6789/0 conn(0x7f6b84151c90 legacy=0x7f6b840cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.870 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.871+0000 7f6b7bfff640 1 -- 172.21.15.70:0/721661263 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b84072d90 con 0x7f6b84152070 2024-04-22T07:03:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.872+0000 7f6b7bfff640 1 --2- 172.21.15.70:0/721661263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b84152070 0x7f6b8415b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097a0 tx=0x7f6b6c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa646b24a3f2aae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.872+0000 7f6b82ffd640 1 -- 172.21.15.70:0/721661263 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c034070 con 0x7f6b84152070 2024-04-22T07:03:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.872+0000 7f6b82ffd640 1 -- 172.21.15.70:0/721661263 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b6c030610 con 0x7f6b84152070 2024-04-22T07:03:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.872+0000 7f6b82ffd640 1 -- 172.21.15.70:0/721661263 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0309e0 con 0x7f6b84152070 2024-04-22T07:03:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.872+0000 7f6b89e77640 1 -- 172.21.15.70:0/721661263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b84152070 msgr2=0x7f6b8415b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.872+0000 7f6b89e77640 1 --2- 172.21.15.70:0/721661263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b84152070 0x7f6b8415b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097a0 tx=0x7f6b6c02b960 comp rx=0 tx=0).stop 2024-04-22T07:03:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.872+0000 7f6b89e77640 1 -- 172.21.15.70:0/721661263 shutdown_connections 2024-04-22T07:03:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.872+0000 7f6b89e77640 1 --2- 172.21.15.70:0/721661263 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b84152070 0x7f6b8415b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.872+0000 7f6b89e77640 1 -- 172.21.15.70:0/721661263 >> 172.21.15.70:0/721661263 conn(0x7f6b840cd4a0 msgr2=0x7f6b840cd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:50.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.872+0000 7f6b89e77640 1 -- 172.21.15.70:0/721661263 shutdown_connections 2024-04-22T07:03:50.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.873+0000 7f6b89e77640 1 -- 172.21.15.70:0/721661263 wait complete. 2024-04-22T07:03:50.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b89e77640 1 Processor -- start 2024-04-22T07:03:50.873 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b89e77640 1 -- start start 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b89e77640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b8406cd30 0x7f6b841574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b89e77640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b84151c90 0x7f6b84149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b89e77640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b84152070 0x7f6b841f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b89e77640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6b840c4920 con 0x7f6b84152070 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b89e77640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84073350 con 0x7f6b8406cd30 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b89e77640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6b84073ab0 con 0x7f6b84151c90 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b8406cd30 0x7f6b841574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b7bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b84151c90 0x7f6b84149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b8406cd30 0x7f6b841574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48572/0 (socket says 172.21.15.70:48572) 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b837fe640 1 -- 172.21.15.70:0/4086918345 learned_addr learned my addr 172.21.15.70:0/4086918345 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.874+0000 7f6b83fff640 1 --2- 172.21.15.70:0/4086918345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b84152070 0x7f6b841f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b837fe640 1 -- 172.21.15.70:0/4086918345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b84151c90 msgr2=0x7f6b84149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b837fe640 1 --2- 172.21.15.70:0/4086918345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b84151c90 0x7f6b84149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b837fe640 1 -- 172.21.15.70:0/4086918345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b84152070 msgr2=0x7f6b841f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b837fe640 1 --2- 172.21.15.70:0/4086918345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b84152070 0x7f6b841f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b837fe640 1 -- 172.21.15.70:0/4086918345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b84153c80 con 0x7f6b8406cd30 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b7bfff640 1 --2- 172.21.15.70:0/4086918345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b84151c90 0x7f6b84149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b837fe640 1 --2- 172.21.15.70:0/4086918345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b8406cd30 0x7f6b841574c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400c560 tx=0x7f6b7400ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b80ff9640 1 -- 172.21.15.70:0/4086918345 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b740052d0 con 0x7f6b8406cd30 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b80ff9640 1 -- 172.21.15.70:0/4086918345 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b74005490 con 0x7f6b8406cd30 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b89e77640 1 -- 172.21.15.70:0/4086918345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b841539a0 con 0x7f6b8406cd30 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b80ff9640 1 -- 172.21.15.70:0/4086918345 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74012870 con 0x7f6b8406cd30 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.875+0000 7f6b89e77640 1 -- 172.21.15.70:0/4086918345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b84154b30 con 0x7f6b8406cd30 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.876+0000 7f6b80ff9640 1 -- 172.21.15.70:0/4086918345 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6b74012a10 con 0x7f6b8406cd30 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.876+0000 7f6b80ff9640 1 -- 172.21.15.70:0/4086918345 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f6b74003be0 con 0x7f6b8406cd30 2024-04-22T07:03:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.876+0000 7f6b89e77640 1 -- 172.21.15.70:0/4086918345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b840722b0 con 0x7f6b8406cd30 2024-04-22T07:03:50.878 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.878+0000 7f6b80ff9640 1 -- 172.21.15.70:0/4086918345 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6b840722b0 con 0x7f6b8406cd30 2024-04-22T07:03:50.919 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:50.921+0000 7f74af443040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:03:50.949 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.951+0000 7f6b89e77640 1 -- 172.21.15.70:0/4086918345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e6d3626b-9227-40fc-af5d-ff49b5da155f", "id": 4} v 0) -- 0x7f6b84069210 con 0x7f6b8406cd30 2024-04-22T07:03:50.954 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.956+0000 7f6b80ff9640 1 -- 172.21.15.70:0/4086918345 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e6d3626b-9227-40fc-af5d-ff49b5da155f", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f6b740035f0 con 0x7f6b8406cd30 2024-04-22T07:03:50.955 INFO:teuthology.orchestra.run.smithi070.stdout:4 2024-04-22T07:03:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.957+0000 7f6b89e77640 1 -- 172.21.15.70:0/4086918345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b8406cd30 msgr2=0x7f6b841574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.957+0000 7f6b89e77640 1 --2- 172.21.15.70:0/4086918345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b8406cd30 0x7f6b841574c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400c560 tx=0x7f6b7400ca20 comp rx=0 tx=0).stop 2024-04-22T07:03:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.957+0000 7f6b89e77640 1 -- 172.21.15.70:0/4086918345 shutdown_connections 2024-04-22T07:03:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.957+0000 7f6b89e77640 1 --2- 172.21.15.70:0/4086918345 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6b84151c90 0x7f6b84149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.957+0000 7f6b89e77640 1 --2- 172.21.15.70:0/4086918345 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6b84152070 0x7f6b841f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.957+0000 7f6b89e77640 1 --2- 172.21.15.70:0/4086918345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b8406cd30 0x7f6b841574c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.957+0000 7f6b89e77640 1 -- 172.21.15.70:0/4086918345 >> 172.21.15.70:0/4086918345 conn(0x7f6b840cd4a0 msgr2=0x7f6b8406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.957+0000 7f6b89e77640 1 -- 172.21.15.70:0/4086918345 shutdown_connections 2024-04-22T07:03:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:50.957+0000 7f6b89e77640 1 -- 172.21.15.70:0/4086918345 wait complete. 2024-04-22T07:03:50.964 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:50.966+0000 7f74af443040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:03:50.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 27e16c28-3d8e-451c-8c1f-1a60b1619f6c 5 2024-04-22T07:03:51.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.105+0000 7f298213b640 1 Processor -- start 2024-04-22T07:03:51.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.105+0000 7f298213b640 1 -- start start 2024-04-22T07:03:51.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f298213b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c06d090 0x7f297c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f298213b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f297c0c4400 0x7f297c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f298213b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f297c0c9220 con 0x7f297c151c40 2024-04-22T07:03:51.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f298213b640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f297c0c8f80 con 0x7f297c0c4400 2024-04-22T07:03:51.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f298213b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f297c0c4ac0 con 0x7f297c06d090 2024-04-22T07:03:51.105 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f2972ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c06d090 0x7f297c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f297b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f297c151c40 0x7f297c157a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59252/0 (socket says 172.21.15.70:59252) 2024-04-22T07:03:51.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f297b7fe640 1 -- 172.21.15.70:0/4149539964 learned_addr learned my addr 172.21.15.70:0/4149539964 (peer_addr_for_me v1:172.21.15.70:0/0) 2024-04-22T07:03:51.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f297bfff640 1 --2- 172.21.15.70:0/4149539964 >> v2:172.21.15.44:3300/0 conn(0x7f297c0c4400 0x7f297c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f297affd640 1 -- 172.21.15.70:0/4149539964 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3224829530 0 0) 0x7f297c0c9220 con 0x7f297c151c40 2024-04-22T07:03:51.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f2972ffd640 1 -- 172.21.15.70:0/4149539964 >> v2:172.21.15.44:3300/0 conn(0x7f297c0c4400 msgr2=0x7f297c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:51.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f2972ffd640 1 --2- 172.21.15.70:0/4149539964 >> v2:172.21.15.44:3300/0 conn(0x7f297c0c4400 0x7f297c0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f2972ffd640 1 -- 172.21.15.70:0/4149539964 >> v1:172.21.15.70:6789/0 conn(0x7f297c151c40 legacy=0x7f297c157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f2972ffd640 1 -- 172.21.15.70:0/4149539964 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c0c9d90 con 0x7f297c06d090 2024-04-22T07:03:51.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.106+0000 7f297bfff640 1 --2- 172.21.15.70:0/4149539964 >> v2:172.21.15.44:3300/0 conn(0x7f297c0c4400 0x7f297c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:03:51.106 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.107+0000 7f2972ffd640 1 --2- 172.21.15.70:0/4149539964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c06d090 0x7f297c0c7e90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29640097a0 tx=0x7f296402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91135599d466ecdf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:51.107 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.107+0000 7f297affd640 1 -- 172.21.15.70:0/4149539964 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964035020 con 0x7f297c06d090 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.107+0000 7f297affd640 1 -- 172.21.15.70:0/4149539964 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2964031830 con 0x7f297c06d090 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.107+0000 7f297affd640 1 -- 172.21.15.70:0/4149539964 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964031c00 con 0x7f297c06d090 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.108+0000 7f298213b640 1 -- 172.21.15.70:0/4149539964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c06d090 msgr2=0x7f297c0c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.108+0000 7f298213b640 1 --2- 172.21.15.70:0/4149539964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c06d090 0x7f297c0c7e90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f29640097a0 tx=0x7f296402b8d0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.108+0000 7f298213b640 1 -- 172.21.15.70:0/4149539964 shutdown_connections 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.108+0000 7f298213b640 1 --2- 172.21.15.70:0/4149539964 >> v2:172.21.15.44:3300/0 conn(0x7f297c0c4400 0x7f297c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.108+0000 7f298213b640 1 --2- 172.21.15.70:0/4149539964 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c06d090 0x7f297c0c7e90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.108+0000 7f298213b640 1 -- 172.21.15.70:0/4149539964 >> 172.21.15.70:0/4149539964 conn(0x7f297c0c9f90 msgr2=0x7f297c0ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.108+0000 7f298213b640 1 -- 172.21.15.70:0/4149539964 shutdown_connections 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.108+0000 7f298213b640 1 -- 172.21.15.70:0/4149539964 wait complete. 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.109+0000 7f298213b640 1 Processor -- start 2024-04-22T07:03:51.108 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.109+0000 7f298213b640 1 -- start start 2024-04-22T07:03:51.110 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f298213b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c06d090 0x7f297c0c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f298213b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c0c4400 0x7f297c142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f298213b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f297c151c40 0x7f297c1431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f298213b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f297c0c9870 con 0x7f297c06d090 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f298213b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f297c0c5040 con 0x7f297c151c40 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f298213b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f297c0c9570 con 0x7f297c0c4400 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f297bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f297c151c40 0x7f297c1431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2972ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c0c4400 0x7f297c142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2972ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c0c4400 0x7f297c142c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:42288/0 (socket says 172.21.15.70:42288) 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2972ffd640 1 -- 172.21.15.70:0/2989560769 learned_addr learned my addr 172.21.15.70:0/2989560769 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f297b7fe640 1 --2- 172.21.15.70:0/2989560769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c06d090 0x7f297c0c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2972ffd640 1 -- 172.21.15.70:0/2989560769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f297c151c40 msgr2=0x7f297c1431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2972ffd640 1 --2- 172.21.15.70:0/2989560769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f297c151c40 0x7f297c1431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2972ffd640 1 -- 172.21.15.70:0/2989560769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c06d090 msgr2=0x7f297c0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.111 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2972ffd640 1 --2- 172.21.15.70:0/2989560769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c06d090 0x7f297c0c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2972ffd640 1 -- 172.21.15.70:0/2989560769 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c1f2e50 con 0x7f297c0c4400 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f297bfff640 1 --2- 172.21.15.70:0/2989560769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f297c151c40 0x7f297c1431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2972ffd640 1 --2- 172.21.15.70:0/2989560769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c0c4400 0x7f297c142c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f29640098c0 tx=0x7f29640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2978ff9640 1 -- 172.21.15.70:0/2989560769 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964040070 con 0x7f297c0c4400 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2978ff9640 1 -- 172.21.15.70:0/2989560769 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296403e070 con 0x7f297c0c4400 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f298213b640 1 -- 172.21.15.70:0/2989560769 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c150b60 con 0x7f297c0c4400 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f2978ff9640 1 -- 172.21.15.70:0/2989560769 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2964035040 con 0x7f297c0c4400 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.110+0000 7f298213b640 1 -- 172.21.15.70:0/2989560769 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c1505a0 con 0x7f297c0c4400 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.111+0000 7f2978ff9640 1 -- 172.21.15.70:0/2989560769 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2964038020 con 0x7f297c0c4400 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.111+0000 7f2978ff9640 1 -- 172.21.15.70:0/2989560769 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f296403ac40 con 0x7f297c0c4400 2024-04-22T07:03:51.112 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.111+0000 7f298213b640 1 -- 172.21.15.70:0/2989560769 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297c04fd90 con 0x7f297c0c4400 2024-04-22T07:03:51.113 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.113+0000 7f2978ff9640 1 -- 172.21.15.70:0/2989560769 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f297c04fd90 con 0x7f297c0c4400 2024-04-22T07:03:51.138 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:51.140+0000 7f74af443040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:03:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.186+0000 7f298213b640 1 -- 172.21.15.70:0/2989560769 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd new", "uuid": "27e16c28-3d8e-451c-8c1f-1a60b1619f6c", "id": 5} v 0) -- 0x7f297c069210 con 0x7f297c0c4400 2024-04-22T07:03:51.187 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:51.189+0000 7f74af443040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:03:51.189 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.191+0000 7f2978ff9640 1 -- 172.21.15.70:0/2989560769 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "27e16c28-3d8e-451c-8c1f-1a60b1619f6c", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f29640317d0 con 0x7f297c0c4400 2024-04-22T07:03:51.189 INFO:teuthology.orchestra.run.smithi070.stdout:5 2024-04-22T07:03:51.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.192+0000 7f298213b640 1 -- 172.21.15.70:0/2989560769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c0c4400 msgr2=0x7f297c142c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.192+0000 7f298213b640 1 --2- 172.21.15.70:0/2989560769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c0c4400 0x7f297c142c90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f29640098c0 tx=0x7f29640046c0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.190 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.192+0000 7f298213b640 1 -- 172.21.15.70:0/2989560769 shutdown_connections 2024-04-22T07:03:51.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.192+0000 7f298213b640 1 --2- 172.21.15.70:0/2989560769 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f297c0c4400 0x7f297c142c90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.192+0000 7f298213b640 1 --2- 172.21.15.70:0/2989560769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f297c151c40 0x7f297c1431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.192+0000 7f298213b640 1 --2- 172.21.15.70:0/2989560769 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f297c06d090 0x7f297c0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.192+0000 7f298213b640 1 -- 172.21.15.70:0/2989560769 >> 172.21.15.70:0/2989560769 conn(0x7f297c0c9f90 msgr2=0x7f297c073060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:51.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.192+0000 7f298213b640 1 -- 172.21.15.70:0/2989560769 shutdown_connections 2024-04-22T07:03:51.191 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.192+0000 7f298213b640 1 -- 172.21.15.70:0/2989560769 wait complete. 2024-04-22T07:03:51.201 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new 91d9b4da-a081-4f7b-9e3b-34cf81c301bf 6 2024-04-22T07:03:51.242 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:51.245+0000 7f74af443040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:03:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.300+0000 7f9e68f87640 1 Processor -- start 2024-04-22T07:03:51.299 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.300+0000 7f9e68f87640 1 -- start start 2024-04-22T07:03:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e68f87640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9e641552c0 0x7f9e640b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e68f87640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e640caec0 0x7f9e640d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e68f87640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e640cc920 0x7f9e640d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e68f87640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f9e641556a0 con 0x7f9e641552c0 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e68f87640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9e6415b160 con 0x7f9e640cc920 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e68f87640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6415b440 con 0x7f9e640caec0 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e5bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e640caec0 0x7f9e640d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e62d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e640cc920 0x7f9e640d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e5bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e640caec0 0x7f9e640d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:42304/0 (socket says 172.21.15.70:42304) 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e62d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e640cc920 0x7f9e640d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48588/0 (socket says 172.21.15.70:48588) 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e5bfff640 1 -- 172.21.15.70:0/3529184087 learned_addr learned my addr 172.21.15.70:0/3529184087 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e6259c640 1 --2- 172.21.15.70:0/3529184087 >> v2:172.21.15.44:3300/0 conn(0x7f9e641552c0 0x7f9e640b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e5bfff640 1 -- 172.21.15.70:0/3529184087 >> v2:172.21.15.70:3300/0 conn(0x7f9e640cc920 msgr2=0x7f9e640d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e5bfff640 1 --2- 172.21.15.70:0/3529184087 >> v2:172.21.15.70:3300/0 conn(0x7f9e640cc920 0x7f9e640d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e5bfff640 1 -- 172.21.15.70:0/3529184087 >> v2:172.21.15.44:3300/0 conn(0x7f9e641552c0 msgr2=0x7f9e640b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e5bfff640 1 --2- 172.21.15.70:0/3529184087 >> v2:172.21.15.44:3300/0 conn(0x7f9e641552c0 0x7f9e640b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e5bfff640 1 -- 172.21.15.70:0/3529184087 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6415b720 con 0x7f9e640caec0 2024-04-22T07:03:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.301+0000 7f9e62d9d640 1 --2- 172.21.15.70:0/3529184087 >> v2:172.21.15.70:3300/0 conn(0x7f9e640cc920 0x7f9e640d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.302+0000 7f9e5bfff640 1 --2- 172.21.15.70:0/3529184087 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e640caec0 0x7f9e640d0a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e64072370 tx=0x7f9e4c02b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f9835c2658369f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.302+0000 7f9e61d9b640 1 -- 172.21.15.70:0/3529184087 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c035020 con 0x7f9e640caec0 2024-04-22T07:03:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.302+0000 7f9e61d9b640 1 -- 172.21.15.70:0/3529184087 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c031870 con 0x7f9e640caec0 2024-04-22T07:03:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.302+0000 7f9e61d9b640 1 -- 172.21.15.70:0/3529184087 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c031c40 con 0x7f9e640caec0 2024-04-22T07:03:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.302+0000 7f9e68f87640 1 -- 172.21.15.70:0/3529184087 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e640caec0 msgr2=0x7f9e640d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.302+0000 7f9e68f87640 1 --2- 172.21.15.70:0/3529184087 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e640caec0 0x7f9e640d0a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e64072370 tx=0x7f9e4c02b910 comp rx=0 tx=0).stop 2024-04-22T07:03:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.303+0000 7f9e68f87640 1 -- 172.21.15.70:0/3529184087 shutdown_connections 2024-04-22T07:03:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.303+0000 7f9e68f87640 1 --2- 172.21.15.70:0/3529184087 >> v2:172.21.15.70:3300/0 conn(0x7f9e640cc920 0x7f9e640d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.303+0000 7f9e68f87640 1 --2- 172.21.15.70:0/3529184087 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e640caec0 0x7f9e640d0a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.303+0000 7f9e68f87640 1 --2- 172.21.15.70:0/3529184087 >> v2:172.21.15.44:3300/0 conn(0x7f9e641552c0 0x7f9e640b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.303+0000 7f9e68f87640 1 -- 172.21.15.70:0/3529184087 >> 172.21.15.70:0/3529184087 conn(0x7f9e640676b0 msgr2=0x7f9e64067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.303+0000 7f9e68f87640 1 -- 172.21.15.70:0/3529184087 shutdown_connections 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.303+0000 7f9e68f87640 1 -- 172.21.15.70:0/3529184087 wait complete. 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.304+0000 7f9e68f87640 1 Processor -- start 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.304+0000 7f9e68f87640 1 -- start start 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e68f87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e640caec0 0x7f9e64156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e68f87640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cc920 0x7f9e64156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e68f87640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e641552c0 0x7f9e6414fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e68f87640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9e640c6a00 con 0x7f9e640cc920 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e68f87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6415bce0 con 0x7f9e640caec0 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e68f87640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9e640cd310 con 0x7f9e641552c0 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e6259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e640caec0 0x7f9e64156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e6259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e640caec0 0x7f9e64156980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48590/0 (socket says 172.21.15.70:48590) 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e6259c640 1 -- 172.21.15.70:0/3360263592 learned_addr learned my addr 172.21.15.70:0/3360263592 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:51.306 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e62d9d640 1 --2- 172.21.15.70:0/3360263592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e641552c0 0x7f9e6414fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e5bfff640 1 --2- 172.21.15.70:0/3360263592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cc920 0x7f9e64156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e6259c640 1 -- 172.21.15.70:0/3360263592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e641552c0 msgr2=0x7f9e6414fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e6259c640 1 --2- 172.21.15.70:0/3360263592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e641552c0 0x7f9e6414fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e6259c640 1 -- 172.21.15.70:0/3360263592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cc920 msgr2=0x7f9e64156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e6259c640 1 --2- 172.21.15.70:0/3360263592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cc920 0x7f9e64156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e6259c640 1 -- 172.21.15.70:0/3360263592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e641f3af0 con 0x7f9e640caec0 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e6259c640 1 --2- 172.21.15.70:0/3360263592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e640caec0 0x7f9e64156980 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400e870 tx=0x7f9e5400ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e5affd640 1 -- 172.21.15.70:0/3360263592 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54017070 con 0x7f9e640caec0 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e5affd640 1 -- 172.21.15.70:0/3360263592 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5400ee60 con 0x7f9e640caec0 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e5affd640 1 -- 172.21.15.70:0/3360263592 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5400cb80 con 0x7f9e640caec0 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e68f87640 1 -- 172.21.15.70:0/3360263592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e641558e0 con 0x7f9e640caec0 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.305+0000 7f9e68f87640 1 -- 172.21.15.70:0/3360263592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e64155ea0 con 0x7f9e640caec0 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.306+0000 7f9e5affd640 1 -- 172.21.15.70:0/3360263592 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e5401a070 con 0x7f9e640caec0 2024-04-22T07:03:51.307 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.306+0000 7f9e5affd640 1 -- 172.21.15.70:0/3360263592 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f9e5400cd20 con 0x7f9e640caec0 2024-04-22T07:03:51.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.306+0000 7f9e68f87640 1 -- 172.21.15.70:0/3360263592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e64068b00 con 0x7f9e640caec0 2024-04-22T07:03:51.308 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.308+0000 7f9e5affd640 1 -- 172.21.15.70:0/3360263592 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9e54013470 con 0x7f9e640caec0 2024-04-22T07:03:51.359 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:51.362+0000 7f74af443040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:03:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.379+0000 7f9e68f87640 1 -- 172.21.15.70:0/3360263592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "91d9b4da-a081-4f7b-9e3b-34cf81c301bf", "id": 6} v 0) -- 0x7f9e640ca970 con 0x7f9e640caec0 2024-04-22T07:03:51.382 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.384+0000 7f9e5affd640 1 -- 172.21.15.70:0/3360263592 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "91d9b4da-a081-4f7b-9e3b-34cf81c301bf", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f9e54013650 con 0x7f9e640caec0 2024-04-22T07:03:51.382 INFO:teuthology.orchestra.run.smithi070.stdout:6 2024-04-22T07:03:51.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.385+0000 7f9e68f87640 1 -- 172.21.15.70:0/3360263592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e640caec0 msgr2=0x7f9e64156980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.385+0000 7f9e68f87640 1 --2- 172.21.15.70:0/3360263592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e640caec0 0x7f9e64156980 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e5400e870 tx=0x7f9e5400ec70 comp rx=0 tx=0).stop 2024-04-22T07:03:51.383 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.385+0000 7f9e68f87640 1 -- 172.21.15.70:0/3360263592 shutdown_connections 2024-04-22T07:03:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.385+0000 7f9e68f87640 1 --2- 172.21.15.70:0/3360263592 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9e641552c0 0x7f9e6414fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.385+0000 7f9e68f87640 1 --2- 172.21.15.70:0/3360263592 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9e640cc920 0x7f9e64156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.385+0000 7f9e68f87640 1 --2- 172.21.15.70:0/3360263592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e640caec0 0x7f9e64156980 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.385+0000 7f9e68f87640 1 -- 172.21.15.70:0/3360263592 >> 172.21.15.70:0/3360263592 conn(0x7f9e640676b0 msgr2=0x7f9e640cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.385+0000 7f9e68f87640 1 -- 172.21.15.70:0/3360263592 shutdown_connections 2024-04-22T07:03:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.385+0000 7f9e68f87640 1 -- 172.21.15.70:0/3360263592 wait complete. 2024-04-22T07:03:51.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd new cec9c5a6-96c3-4a2e-8e53-bc69804ff685 7 2024-04-22T07:03:51.492 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.494+0000 7fc0d9fb2640 1 Processor -- start 2024-04-22T07:03:51.493 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.494+0000 7fc0d9fb2640 1 -- start start 2024-04-22T07:03:51.494 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.494+0000 7fc0d9fb2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0d4151c90 0x7fc0d40cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.494 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.494+0000 7fc0d9fb2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d415bb40 0x7fc0d40ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.494+0000 7fc0d9fb2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0d40c4c30 con 0x7fc0d4152070 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.494+0000 7fc0d9fb2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc0d40c4ef0 con 0x7fc0d4151c90 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.494+0000 7fc0d9fb2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d4072ab0 con 0x7fc0d415bb40 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0d4151c90 0x7fc0d40cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d3fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d415bb40 0x7fc0d40ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d3fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d415bb40 0x7fc0d40ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.70:42324/0 (socket says 172.21.15.70:42324) 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d2ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc0d4152070 0x7fc0d415b430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59264/0 (socket says 172.21.15.70:59264) 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d3fff640 1 -- 172.21.15.70:0/3474656049 learned_addr learned my addr 172.21.15.70:0/3474656049 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0d4151c90 0x7fc0d40cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48606/0 (socket says 172.21.15.70:48606) 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d37fe640 1 -- 172.21.15.70:0/3474656049 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d415bb40 msgr2=0x7fc0d40ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d37fe640 1 --2- 172.21.15.70:0/3474656049 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d415bb40 0x7fc0d40ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d37fe640 1 -- 172.21.15.70:0/3474656049 >> v1:172.21.15.70:6789/0 conn(0x7fc0d4152070 legacy=0x7fc0d415b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.495 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d37fe640 1 -- 172.21.15.70:0/3474656049 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d4072d90 con 0x7fc0d4151c90 2024-04-22T07:03:51.496 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d27fc640 1 -- 172.21.15.70:0/3474656049 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 643150267 0 0) 0x7fc0d40c4c30 con 0x7fc0d4152070 2024-04-22T07:03:51.496 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.495+0000 7fc0d3fff640 1 --2- 172.21.15.70:0/3474656049 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d415bb40 0x7fc0d40ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:51.496 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.496+0000 7fc0d37fe640 1 --2- 172.21.15.70:0/3474656049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4151c90 0x7fc0d40cc960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40097a0 tx=0x7fc0c402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9d87370fba80414 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:51.496 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.496+0000 7fc0d27fc640 1 -- 172.21.15.70:0/3474656049 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4034070 con 0x7fc0d4151c90 2024-04-22T07:03:51.496 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.496+0000 7fc0d27fc640 1 -- 172.21.15.70:0/3474656049 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c4030610 con 0x7fc0d4151c90 2024-04-22T07:03:51.496 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.496+0000 7fc0d27fc640 1 -- 172.21.15.70:0/3474656049 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40309e0 con 0x7fc0d4151c90 2024-04-22T07:03:51.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.496+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/3474656049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4151c90 msgr2=0x7fc0d40cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.496+0000 7fc0d9fb2640 1 --2- 172.21.15.70:0/3474656049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4151c90 0x7fc0d40cc960 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40097a0 tx=0x7fc0c402b960 comp rx=0 tx=0).stop 2024-04-22T07:03:51.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.496+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/3474656049 shutdown_connections 2024-04-22T07:03:51.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.497+0000 7fc0d9fb2640 1 --2- 172.21.15.70:0/3474656049 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d415bb40 0x7fc0d40ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.497+0000 7fc0d9fb2640 1 --2- 172.21.15.70:0/3474656049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d4151c90 0x7fc0d40cc960 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.497+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/3474656049 >> 172.21.15.70:0/3474656049 conn(0x7fc0d40cd4a0 msgr2=0x7fc0d40cd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:51.499 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.497+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/3474656049 shutdown_connections 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.497+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/3474656049 wait complete. 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.498+0000 7fc0d9fb2640 1 Processor -- start 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.498+0000 7fc0d9fb2640 1 -- start start 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.498+0000 7fc0d9fb2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0d4151c90 0x7fc0d41e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.498+0000 7fc0d9fb2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d4152070 0x7fc0d414c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.498+0000 7fc0d9fb2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d415bb40 0x7fc0d41e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.498+0000 7fc0d9fb2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d40c4920 con 0x7fc0d4151c90 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.498+0000 7fc0d9fb2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4073350 con 0x7fc0d415bb40 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.498+0000 7fc0d9fb2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d4073ab0 con 0x7fc0d4152070 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d415bb40 0x7fc0d41e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d415bb40 0x7fc0d41e0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48608/0 (socket says 172.21.15.70:48608) 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d3fff640 1 -- 172.21.15.70:0/1858674896 learned_addr learned my addr 172.21.15.70:0/1858674896 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d2ffd640 1 --2- 172.21.15.70:0/1858674896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d4152070 0x7fc0d414c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d37fe640 1 --2- 172.21.15.70:0/1858674896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0d4151c90 0x7fc0d41e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.500 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d3fff640 1 -- 172.21.15.70:0/1858674896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d4152070 msgr2=0x7fc0d414c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d3fff640 1 --2- 172.21.15.70:0/1858674896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d4152070 0x7fc0d414c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d3fff640 1 -- 172.21.15.70:0/1858674896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0d4151c90 msgr2=0x7fc0d41e2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d3fff640 1 --2- 172.21.15.70:0/1858674896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0d4151c90 0x7fc0d41e2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d3fff640 1 -- 172.21.15.70:0/1858674896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d41e3410 con 0x7fc0d415bb40 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d2ffd640 1 --2- 172.21.15.70:0/1858674896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d4152070 0x7fc0d414c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d3fff640 1 --2- 172.21.15.70:0/1858674896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d415bb40 0x7fc0d41e0cc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009e50 tx=0x7fc0cc00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0bbfff640 1 -- 172.21.15.70:0/1858674896 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc00cb80 con 0x7fc0d415bb40 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0bbfff640 1 -- 172.21.15.70:0/1858674896 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0cc00cd20 con 0x7fc0d415bb40 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0bbfff640 1 -- 172.21.15.70:0/1858674896 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc003bb0 con 0x7fc0d415bb40 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/1858674896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d415afd0 con 0x7fc0d415bb40 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/1858674896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d41e3960 con 0x7fc0d415bb40 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0bbfff640 1 -- 172.21.15.70:0/1858674896 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc0cc01b030 con 0x7fc0d415bb40 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.499+0000 7fc0bbfff640 1 -- 172.21.15.70:0/1858674896 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fc0cc01dca0 con 0x7fc0d415bb40 2024-04-22T07:03:51.501 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.500+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/1858674896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d4069210 con 0x7fc0d415bb40 2024-04-22T07:03:51.502 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.502+0000 7fc0bbfff640 1 -- 172.21.15.70:0/1858674896 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc0cc022020 con 0x7fc0d415bb40 2024-04-22T07:03:51.573 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.575+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/1858674896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cec9c5a6-96c3-4a2e-8e53-bc69804ff685", "id": 7} v 0) -- 0x7fc0d40c6ce0 con 0x7fc0d415bb40 2024-04-22T07:03:51.578 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.580+0000 7fc0bbfff640 1 -- 172.21.15.70:0/1858674896 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cec9c5a6-96c3-4a2e-8e53-bc69804ff685", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fc0cc010040 con 0x7fc0d415bb40 2024-04-22T07:03:51.578 INFO:teuthology.orchestra.run.smithi070.stdout:7 2024-04-22T07:03:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.581+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/1858674896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d415bb40 msgr2=0x7fc0d41e0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.581+0000 7fc0d9fb2640 1 --2- 172.21.15.70:0/1858674896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d415bb40 0x7fc0d41e0cc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009e50 tx=0x7fc0cc00c490 comp rx=0 tx=0).stop 2024-04-22T07:03:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.581+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/1858674896 shutdown_connections 2024-04-22T07:03:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.581+0000 7fc0d9fb2640 1 --2- 172.21.15.70:0/1858674896 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc0d4152070 0x7fc0d414c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.581+0000 7fc0d9fb2640 1 --2- 172.21.15.70:0/1858674896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0d415bb40 0x7fc0d41e0cc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.581+0000 7fc0d9fb2640 1 --2- 172.21.15.70:0/1858674896 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc0d4151c90 0x7fc0d41e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.581+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/1858674896 >> 172.21.15.70:0/1858674896 conn(0x7fc0d40cd4a0 msgr2=0x7fc0d41493b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.581+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/1858674896 shutdown_connections 2024-04-22T07:03:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:2024-04-22T07:03:51.581+0000 7fc0d9fb2640 1 -- 172.21.15.70:0/1858674896 wait complete. 2024-04-22T07:03:51.590 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-22T07:03:51.590 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-22T07:03:51.592 INFO:tasks.ceph.osd.0:Started 2024-04-22T07:03:51.592 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-22T07:03:51.592 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-22T07:03:51.596 INFO:tasks.ceph.osd.1:Started 2024-04-22T07:03:51.596 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-22T07:03:51.597 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-22T07:03:51.598 INFO:tasks.ceph.osd.2:Started 2024-04-22T07:03:51.598 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-22T07:03:51.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-22T07:03:51.601 INFO:tasks.ceph.osd.3:Started 2024-04-22T07:03:51.601 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-22T07:03:51.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-22T07:03:51.603 INFO:tasks.ceph.osd.4:Started 2024-04-22T07:03:51.603 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-22T07:03:51.603 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-22T07:03:51.605 INFO:tasks.ceph.osd.5:Started 2024-04-22T07:03:51.605 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-22T07:03:51.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-22T07:03:51.607 INFO:tasks.ceph.osd.6:Started 2024-04-22T07:03:51.607 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-22T07:03:51.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-22T07:03:51.610 INFO:tasks.ceph.osd.7:Started 2024-04-22T07:03:51.610 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:03:51.623 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:51.625+0000 7f2de9133040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T07:03:51.730 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:51.731+0000 7f2de9133040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T07:03:51.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.741+0000 7fd57c3e6640 1 Processor -- start 2024-04-22T07:03:51.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.741+0000 7fd57c3e6640 1 -- start start 2024-04-22T07:03:51.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57c3e6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5740cbbf0 0x7fd574072130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57c3e6640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd5740ca820 0x7fd574154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57c3e6640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5740cc180 con 0x7fd5740cb810 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57c3e6640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd574072670 con 0x7fd5740ca820 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57c3e6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd5740cc420 con 0x7fd5740cbbf0 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57a983640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd5740ca820 0x7fd574154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57a182640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd5740cb810 0x7fd57415ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46352/0 (socket says 172.21.15.44:46352) 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57a182640 1 -- 172.21.15.44:0/1158266736 learned_addr learned my addr 172.21.15.44:0/1158266736 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd579981640 1 --2- 172.21.15.44:0/1158266736 >> v2:172.21.15.70:3300/0 conn(0x7fd5740cbbf0 0x7fd574072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57a983640 1 -- 172.21.15.44:0/1158266736 >> v2:172.21.15.70:3300/0 conn(0x7fd5740cbbf0 msgr2=0x7fd574072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57a983640 1 --2- 172.21.15.44:0/1158266736 >> v2:172.21.15.70:3300/0 conn(0x7fd5740cbbf0 0x7fd574072130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.742+0000 7fd57a983640 1 -- 172.21.15.44:0/1158266736 >> v1:172.21.15.44:6789/0 conn(0x7fd5740cb810 legacy=0x7fd57415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.743+0000 7fd57a983640 1 -- 172.21.15.44:0/1158266736 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5740cc700 con 0x7fd5740ca820 2024-04-22T07:03:51.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.743+0000 7fd579180640 1 -- 172.21.15.44:0/1158266736 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4106624181 0 0) 0x7fd5740cc180 con 0x7fd5740cb810 2024-04-22T07:03:51.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.743+0000 7fd57a983640 1 --2- 172.21.15.44:0/1158266736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 0x7fd574154ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009d90 tx=0x7fd56c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b898f9d3a422e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:51.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.743+0000 7fd579180640 1 -- 172.21.15.44:0/1158266736 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c039070 con 0x7fd5740ca820 2024-04-22T07:03:51.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.744+0000 7fd579180640 1 -- 172.21.15.44:0/1158266736 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c02cd00 con 0x7fd5740ca820 2024-04-22T07:03:51.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.744+0000 7fd579180640 1 -- 172.21.15.44:0/1158266736 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c034660 con 0x7fd5740ca820 2024-04-22T07:03:51.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.744+0000 7fd57c3e6640 1 -- 172.21.15.44:0/1158266736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 msgr2=0x7fd574154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.744+0000 7fd57c3e6640 1 --2- 172.21.15.44:0/1158266736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 0x7fd574154ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009d90 tx=0x7fd56c02bf50 comp rx=0 tx=0).stop 2024-04-22T07:03:51.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.744+0000 7fd57c3e6640 1 -- 172.21.15.44:0/1158266736 shutdown_connections 2024-04-22T07:03:51.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.744+0000 7fd57c3e6640 1 --2- 172.21.15.44:0/1158266736 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 0x7fd574154ab0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.744+0000 7fd57c3e6640 1 --2- 172.21.15.44:0/1158266736 >> v2:172.21.15.70:3300/0 conn(0x7fd5740cbbf0 0x7fd574072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.744+0000 7fd57c3e6640 1 -- 172.21.15.44:0/1158266736 >> 172.21.15.44:0/1158266736 conn(0x7fd57406b970 msgr2=0x7fd57406bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:51.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.744+0000 7fd57c3e6640 1 -- 172.21.15.44:0/1158266736 shutdown_connections 2024-04-22T07:03:51.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.744+0000 7fd57c3e6640 1 -- 172.21.15.44:0/1158266736 wait complete. 2024-04-22T07:03:51.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.745+0000 7fd57c3e6640 1 Processor -- start 2024-04-22T07:03:51.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd57c3e6640 1 -- start start 2024-04-22T07:03:51.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd57c3e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 0x7fd5741f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd57c3e6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5740cb810 0x7fd57414af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd57c3e6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5740cbbf0 0x7fd57414d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd57c3e6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd574072910 con 0x7fd5740ca820 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd57c3e6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd574072f40 con 0x7fd5740cb810 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd57c3e6640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd574073460 con 0x7fd5740cbbf0 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd57a182640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 0x7fd5741f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd579981640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5740cb810 0x7fd57414af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd57a182640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 0x7fd5741f0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33724/0 (socket says 172.21.15.44:33724) 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.746+0000 7fd57a182640 1 -- 172.21.15.44:0/998671992 learned_addr learned my addr 172.21.15.44:0/998671992 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd57a983640 1 --2- 172.21.15.44:0/998671992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5740cbbf0 0x7fd57414d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd57a182640 1 -- 172.21.15.44:0/998671992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5740cbbf0 msgr2=0x7fd57414d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd57a182640 1 --2- 172.21.15.44:0/998671992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5740cbbf0 0x7fd57414d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd57a182640 1 -- 172.21.15.44:0/998671992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5740cb810 msgr2=0x7fd57414af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd57a182640 1 --2- 172.21.15.44:0/998671992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5740cb810 0x7fd57414af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd57a182640 1 -- 172.21.15.44:0/998671992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd574149030 con 0x7fd5740ca820 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd57a182640 1 --2- 172.21.15.44:0/998671992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 0x7fd5741f0b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c4d0 tx=0x7fd56400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd56affd640 1 -- 172.21.15.44:0/998671992 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564018070 con 0x7fd5740ca820 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd56affd640 1 -- 172.21.15.44:0/998671992 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5640074e0 con 0x7fd5740ca820 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd57c3e6640 1 -- 172.21.15.44:0/998671992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd574147d90 con 0x7fd5740ca820 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd56affd640 1 -- 172.21.15.44:0/998671992 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564004a20 con 0x7fd5740ca820 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.747+0000 7fd57c3e6640 1 -- 172.21.15.44:0/998671992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd574148a70 con 0x7fd5740ca820 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.748+0000 7fd56affd640 1 -- 172.21.15.44:0/998671992 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd56401b070 con 0x7fd5740ca820 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.748+0000 7fd56affd640 1 -- 172.21.15.44:0/998671992 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd564012ad0 con 0x7fd5740ca820 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.748+0000 7fd57c3e6640 1 -- 172.21.15.44:0/998671992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd574073ec0 con 0x7fd5740ca820 2024-04-22T07:03:51.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.750+0000 7fd56affd640 1 -- 172.21.15.44:0/998671992 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd56401d440 con 0x7fd5740ca820 2024-04-22T07:03:51.772 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:51.774+0000 7f2de9133040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T07:03:51.791 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:51.793+0000 7f74af443040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:03:51.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.813+0000 7fd57c3e6640 1 -- 172.21.15.44:0/998671992 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd57404fdc0 con 0x7fd5740ca820 2024-04-22T07:03:51.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.814+0000 7fd56affd640 1 -- 172.21.15.44:0/998671992 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd56401d620 con 0x7fd5740ca820 2024-04-22T07:03:51.812 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:03:51.812 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":10,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:03:51.577561+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:03:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.815+0000 7fd57c3e6640 1 -- 172.21.15.44:0/998671992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 msgr2=0x7fd5741f0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.815+0000 7fd57c3e6640 1 --2- 172.21.15.44:0/998671992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 0x7fd5741f0b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c4d0 tx=0x7fd56400c990 comp rx=0 tx=0).stop 2024-04-22T07:03:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.815+0000 7fd57c3e6640 1 -- 172.21.15.44:0/998671992 shutdown_connections 2024-04-22T07:03:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.815+0000 7fd57c3e6640 1 --2- 172.21.15.44:0/998671992 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd5740cbbf0 0x7fd57414d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.815+0000 7fd57c3e6640 1 --2- 172.21.15.44:0/998671992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5740cb810 0x7fd57414af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.815+0000 7fd57c3e6640 1 --2- 172.21.15.44:0/998671992 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd5740ca820 0x7fd5741f0b50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.815+0000 7fd57c3e6640 1 -- 172.21.15.44:0/998671992 >> 172.21.15.44:0/998671992 conn(0x7fd57406b970 msgr2=0x7fd5741e64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.815+0000 7fd57c3e6640 1 -- 172.21.15.44:0/998671992 shutdown_connections 2024-04-22T07:03:51.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:03:51.815+0000 7fd57c3e6640 1 -- 172.21.15.44:0/998671992 wait complete. 2024-04-22T07:03:51.823 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-22T07:03:51.823 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-22T07:03:51.938 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:51.939+0000 7f2de9133040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T07:03:51.991 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:51.992+0000 7f2de9133040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T07:03:52.043 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:52.045+0000 7f2de9133040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T07:03:52.062 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:52.065+0000 7f74af443040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:03:52.117 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:52.120+0000 7f74af443040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:03:52.153 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:52.155+0000 7f2de9133040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T07:03:52.207 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T07:03:52.209+0000 7fecfd4b5640 -1 Falling back to public interface 2024-04-22T07:03:52.210 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T07:03:52.212+0000 7fb9e53a7640 -1 Falling back to public interface 2024-04-22T07:03:52.212 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T07:03:52.214+0000 7fcf61d8e640 -1 Falling back to public interface 2024-04-22T07:03:52.221 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T07:03:52.222+0000 7fab961f8640 -1 Falling back to public interface 2024-04-22T07:03:52.221 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T07:03:52.222+0000 7f6f37c0c640 -1 Falling back to public interface 2024-04-22T07:03:52.222 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T07:03:52.224+0000 7fb5c4c1a640 -1 Falling back to public interface 2024-04-22T07:03:52.229 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T07:03:52.231+0000 7f123a00c640 -1 Falling back to public interface 2024-04-22T07:03:52.232 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T07:03:52.235+0000 7f82c1296640 -1 Falling back to public interface 2024-04-22T07:03:52.244 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:52.246+0000 7f74af443040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:03:52.537 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:52.539+0000 7f2de9133040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T07:03:52.539 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:52.541+0000 7f74af443040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:03:52.588 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:52.591+0000 7f74af443040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:03:52.629 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T07:03:52.722 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.724+0000 7ffa9dd7e640 1 Processor -- start 2024-04-22T07:03:52.722 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.724+0000 7ffa9dd7e640 1 -- start start 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa9dd7e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa980c6c80 0x7ffa980ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa9dd7e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ffa980c7060 0x7ffa9815bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa9dd7e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa980c7e10 con 0x7ffa9815c4e0 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa9dd7e640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7ffa980c9630 con 0x7ffa980c7060 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa9dd7e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ffa980c80b0 con 0x7ffa980c6c80 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa97fff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ffa9815c4e0 0x7ffa981526b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:46358/0 (socket says 172.21.15.44:46358) 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa97fff640 1 -- 172.21.15.44:0/1534680103 learned_addr learned my addr 172.21.15.44:0/1534680103 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa96ffd640 1 --2- 172.21.15.44:0/1534680103 >> v2:172.21.15.44:3300/0 conn(0x7ffa980c7060 0x7ffa9815bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa977fe640 1 --2- 172.21.15.44:0/1534680103 >> v2:172.21.15.70:3300/0 conn(0x7ffa980c6c80 0x7ffa980ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa967fc640 1 -- 172.21.15.44:0/1534680103 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587617312 0 0) 0x7ffa980c7e10 con 0x7ffa9815c4e0 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.725+0000 7ffa967fc640 1 -- 172.21.15.44:0/1534680103 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa80003520 con 0x7ffa9815c4e0 2024-04-22T07:03:52.724 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.726+0000 7ffa977fe640 1 -- 172.21.15.44:0/1534680103 >> v2:172.21.15.44:3300/0 conn(0x7ffa980c7060 msgr2=0x7ffa9815bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:52.725 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.726+0000 7ffa977fe640 1 --2- 172.21.15.44:0/1534680103 >> v2:172.21.15.44:3300/0 conn(0x7ffa980c7060 0x7ffa9815bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:52.725 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.726+0000 7ffa977fe640 1 -- 172.21.15.44:0/1534680103 >> v1:172.21.15.44:6789/0 conn(0x7ffa9815c4e0 legacy=0x7ffa981526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:52.725 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.726+0000 7ffa977fe640 1 -- 172.21.15.44:0/1534680103 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa98070750 con 0x7ffa980c6c80 2024-04-22T07:03:52.725 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.726+0000 7ffa96ffd640 1 --2- 172.21.15.44:0/1534680103 >> v2:172.21.15.44:3300/0 conn(0x7ffa980c7060 0x7ffa9815bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:52.725 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.726+0000 7ffa977fe640 1 --2- 172.21.15.44:0/1534680103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa980c6c80 0x7ffa980ca600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffa980cd260 tx=0x7ffa8802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd9b9b79ef8e358a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:52.729 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.726+0000 7ffa967fc640 1 -- 172.21.15.44:0/1534680103 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88034070 con 0x7ffa980c6c80 2024-04-22T07:03:52.729 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.727+0000 7ffa967fc640 1 -- 172.21.15.44:0/1534680103 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa88030610 con 0x7ffa980c6c80 2024-04-22T07:03:52.729 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.727+0000 7ffa967fc640 1 -- 172.21.15.44:0/1534680103 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa880309e0 con 0x7ffa980c6c80 2024-04-22T07:03:52.729 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.727+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/1534680103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa980c6c80 msgr2=0x7ffa980ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:52.729 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.727+0000 7ffa9dd7e640 1 --2- 172.21.15.44:0/1534680103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa980c6c80 0x7ffa980ca600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffa980cd260 tx=0x7ffa8802b960 comp rx=0 tx=0).stop 2024-04-22T07:03:52.729 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.727+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/1534680103 shutdown_connections 2024-04-22T07:03:52.729 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.727+0000 7ffa9dd7e640 1 --2- 172.21.15.44:0/1534680103 >> v2:172.21.15.44:3300/0 conn(0x7ffa980c7060 0x7ffa9815bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.727+0000 7ffa9dd7e640 1 --2- 172.21.15.44:0/1534680103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa980c6c80 0x7ffa980ca600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.727+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/1534680103 >> 172.21.15.44:0/1534680103 conn(0x7ffa98072760 msgr2=0x7ffa98072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.727+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/1534680103 shutdown_connections 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.727+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/1534680103 wait complete. 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.728+0000 7ffa9dd7e640 1 Processor -- start 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.728+0000 7ffa9dd7e640 1 -- start start 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa9dd7e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa980c6c80 0x7ffa980cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa9dd7e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa980c7060 0x7ffa98147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa9dd7e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9815c4e0 0x7ffa980cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa9dd7e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ffa980c9a50 con 0x7ffa980c6c80 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa9dd7e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffa98070d10 con 0x7ffa9815c4e0 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa9dd7e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ffa9806b910 con 0x7ffa980c7060 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa977fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa980c6c80 0x7ffa980cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa977fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa980c6c80 0x7ffa980cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:33730/0 (socket says 172.21.15.44:33730) 2024-04-22T07:03:52.730 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa977fe640 1 -- 172.21.15.44:0/3696246365 learned_addr learned my addr 172.21.15.44:0/3696246365 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa97fff640 1 --2- 172.21.15.44:0/3696246365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9815c4e0 0x7ffa980cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.729+0000 7ffa96ffd640 1 --2- 172.21.15.44:0/3696246365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa980c7060 0x7ffa98147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa977fe640 1 -- 172.21.15.44:0/3696246365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa980c7060 msgr2=0x7ffa98147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa977fe640 1 --2- 172.21.15.44:0/3696246365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa980c7060 0x7ffa98147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa977fe640 1 -- 172.21.15.44:0/3696246365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9815c4e0 msgr2=0x7ffa980cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa977fe640 1 --2- 172.21.15.44:0/3696246365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9815c4e0 0x7ffa980cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa977fe640 1 -- 172.21.15.44:0/3696246365 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa980c5a40 con 0x7ffa980c6c80 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa97fff640 1 --2- 172.21.15.44:0/3696246365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9815c4e0 0x7ffa980cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa977fe640 1 --2- 172.21.15.44:0/3696246365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa980c6c80 0x7ffa980cd010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffa8802bf70 tx=0x7ffa880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa7ffff640 1 -- 172.21.15.44:0/3696246365 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88034040 con 0x7ffa980c6c80 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa7ffff640 1 -- 172.21.15.44:0/3696246365 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa8803aaa0 con 0x7ffa980c6c80 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa7ffff640 1 -- 172.21.15.44:0/3696246365 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88006e10 con 0x7ffa980c6c80 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/3696246365 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa980c5d20 con 0x7ffa980c6c80 2024-04-22T07:03:52.731 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/3696246365 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa98151690 con 0x7ffa980c6c80 2024-04-22T07:03:52.732 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa7ffff640 1 -- 172.21.15.44:0/3696246365 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffa88037070 con 0x7ffa980c6c80 2024-04-22T07:03:52.732 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.730+0000 7ffa7ffff640 1 -- 172.21.15.44:0/3696246365 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ffa88044c70 con 0x7ffa980c6c80 2024-04-22T07:03:52.732 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.731+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/3696246365 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa98069430 con 0x7ffa980c6c80 2024-04-22T07:03:52.732 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.733+0000 7ffa7ffff640 1 -- 172.21.15.44:0/3696246365 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ffa98069430 con 0x7ffa980c6c80 2024-04-22T07:03:52.756 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:52.758+0000 7f74af443040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:03:52.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.796+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/3696246365 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa9804fce0 con 0x7ffa980c6c80 2024-04-22T07:03:52.794 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.797+0000 7ffa7ffff640 1 -- 172.21.15.44:0/3696246365 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ffa8804a060 con 0x7ffa980c6c80 2024-04-22T07:03:52.795 INFO:teuthology.misc.health.smithi044.stdout: 2024-04-22T07:03:52.795 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":10,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:03:51.577561+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:03:52.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.797+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/3696246365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa980c6c80 msgr2=0x7ffa980cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:52.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.797+0000 7ffa9dd7e640 1 --2- 172.21.15.44:0/3696246365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa980c6c80 0x7ffa980cd010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffa8802bf70 tx=0x7ffa880046c0 comp rx=0 tx=0).stop 2024-04-22T07:03:52.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.798+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/3696246365 shutdown_connections 2024-04-22T07:03:52.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.798+0000 7ffa9dd7e640 1 --2- 172.21.15.44:0/3696246365 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ffa980c7060 0x7ffa98147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:52.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.798+0000 7ffa9dd7e640 1 --2- 172.21.15.44:0/3696246365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9815c4e0 0x7ffa980cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:52.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.798+0000 7ffa9dd7e640 1 --2- 172.21.15.44:0/3696246365 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ffa980c6c80 0x7ffa980cd010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:52.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.798+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/3696246365 >> 172.21.15.44:0/3696246365 conn(0x7ffa98072760 msgr2=0x7ffa9806fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:52.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.798+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/3696246365 shutdown_connections 2024-04-22T07:03:52.796 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:52.798+0000 7ffa9dd7e640 1 -- 172.21.15.44:0/3696246365 wait complete. 2024-04-22T07:03:52.797 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:52.797+0000 7f2de9133040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T07:03:52.805 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-22T07:03:52.847 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:52.849+0000 7f2de9133040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T07:03:52.900 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:52.903+0000 7f74af443040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:03:52.952 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:52.954+0000 7f74af443040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:03:52.963 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:52.964+0000 7f2de9133040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T07:03:53.025 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:53.028+0000 7f74af443040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:03:53.073 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:53.076+0000 7f74af443040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:03:53.241 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:53.242+0000 7f2de9133040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T07:03:53.288 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:53.289+0000 7f2de9133040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T07:03:53.444 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:53.445+0000 7f2de9133040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T07:03:53.575 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:53.578+0000 7f74af443040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:03:53.578 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:53.580+0000 7f2de9133040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T07:03:53.629 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:53.631+0000 7f2de9133040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T07:03:53.697 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:53.699+0000 7f74af443040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:03:53.697 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:53.699+0000 7f2de9133040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T07:03:53.746 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:53.747+0000 7f2de9133040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T07:03:54.220 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:54.221+0000 7f2de9133040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T07:03:54.335 INFO:tasks.ceph.mgr.x.smithi070.stderr:2024-04-22T07:03:54.337+0000 7f2de9133040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T07:03:54.709 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:54.712+0000 7f7450173640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-22T07:03:54.710 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:03:54.712+0000 7f7450173640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-22T07:03:55.663 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T07:03:55.666+0000 7fecfd4b5640 -1 osd.2 0 log_to_monitors true 2024-04-22T07:03:55.730 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T07:03:55.733+0000 7fb5c4c1a640 -1 osd.1 0 log_to_monitors true 2024-04-22T07:03:55.752 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T07:03:55.747+0000 7f6f37c0c640 -1 osd.5 0 log_to_monitors true 2024-04-22T07:03:55.752 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T07:03:55.746+0000 7f82c1296640 -1 osd.0 0 log_to_monitors true 2024-04-22T07:03:55.752 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T07:03:55.747+0000 7fb9e53a7640 -1 osd.6 0 log_to_monitors true 2024-04-22T07:03:55.753 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T07:03:55.752+0000 7fab961f8640 -1 osd.4 0 log_to_monitors true 2024-04-22T07:03:55.753 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T07:03:55.753+0000 7fcf61d8e640 -1 osd.7 0 log_to_monitors true 2024-04-22T07:03:55.753 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T07:03:55.755+0000 7f123a00c640 -1 osd.3 0 log_to_monitors true 2024-04-22T07:03:57.662 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T07:03:57.665+0000 7fecf944b640 -1 osd.2 0 waiting for initial osdmap 2024-04-22T07:03:57.675 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T07:03:57.677+0000 7fecf126b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:59.119 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T07:03:59.121+0000 7f6f3352a640 -1 osd.5 0 waiting for initial osdmap 2024-04-22T07:03:59.129 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T07:03:59.131+0000 7f6f2b34a640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:59.136 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T07:03:59.137+0000 7fcf5dd24640 -1 osd.7 0 waiting for initial osdmap 2024-04-22T07:03:59.142 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T07:03:59.144+0000 7fab9218e640 -1 osd.4 0 waiting for initial osdmap 2024-04-22T07:03:59.151 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T07:03:59.153+0000 7fcf55b44640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:59.153 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T07:03:59.155+0000 7fb9e133d640 -1 osd.6 0 waiting for initial osdmap 2024-04-22T07:03:59.155 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T07:03:59.157+0000 7fab89fae640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:59.171 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T07:03:59.173+0000 7fb9d915d640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:59.386 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T07:03:59.389+0000 7f82bd22c640 -1 osd.0 0 waiting for initial osdmap 2024-04-22T07:03:59.399 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T07:03:59.402+0000 7f82b504c640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:59.452 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T07:03:59.454+0000 7fb5c067d640 -1 osd.1 0 waiting for initial osdmap 2024-04-22T07:03:59.456 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T07:03:59.459+0000 7f12358c1640 -1 osd.3 0 waiting for initial osdmap 2024-04-22T07:03:59.474 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T07:03:59.476+0000 7fb5b849d640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:59.479 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T07:03:59.482+0000 7f122d6e1640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T07:03:59.613 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T07:03:59.707 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.709+0000 7fb243f8f640 1 Processor -- start 2024-04-22T07:03:59.707 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.709+0000 7fb243f8f640 1 -- start start 2024-04-22T07:03:59.708 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb243f8f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb23c0cc6e0 0x7fb23c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.708 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb243f8f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb23c0c5540 0x7fb23c0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.708 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb243f8f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb23c15ef20 con 0x7fb23c1550b0 2024-04-22T07:03:59.708 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb243f8f640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fb23c15bbe0 con 0x7fb23c0cc6e0 2024-04-22T07:03:59.708 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb243f8f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb23c0cb270 con 0x7fb23c0c5540 2024-04-22T07:03:59.709 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb24152a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb23c0cc6e0 0x7fb23c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.709 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb24152a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb23c0cc6e0 0x7fb23c0cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55606/0 (socket says 172.21.15.44:55606) 2024-04-22T07:03:59.709 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb24152a640 1 -- 172.21.15.44:0/1488383725 learned_addr learned my addr 172.21.15.44:0/1488383725 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:03:59.709 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb24252c640 1 --2- 172.21.15.44:0/1488383725 >> v2:172.21.15.70:3300/0 conn(0x7fb23c0c5540 0x7fb23c0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.709 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb24152a640 1 -- 172.21.15.44:0/1488383725 >> v2:172.21.15.70:3300/0 conn(0x7fb23c0c5540 msgr2=0x7fb23c0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:03:59.709 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb24152a640 1 --2- 172.21.15.44:0/1488383725 >> v2:172.21.15.70:3300/0 conn(0x7fb23c0c5540 0x7fb23c0cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.709 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb24152a640 1 -- 172.21.15.44:0/1488383725 >> v1:172.21.15.70:6789/0 conn(0x7fb23c1550b0 legacy=0x7fb23c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.709 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.710+0000 7fb24152a640 1 -- 172.21.15.44:0/1488383725 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c0cb4c0 con 0x7fb23c0cc6e0 2024-04-22T07:03:59.710 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.711+0000 7fb24152a640 1 --2- 172.21.15.44:0/1488383725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c0cc6e0 0x7fb23c0cd230 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0cddb0 tx=0x7fb23002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dfc1baaad1c9fd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:59.710 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.711+0000 7fb240d29640 1 -- 172.21.15.44:0/1488383725 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230035020 con 0x7fb23c0cc6e0 2024-04-22T07:03:59.710 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.711+0000 7fb240d29640 1 -- 172.21.15.44:0/1488383725 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb230031870 con 0x7fb23c0cc6e0 2024-04-22T07:03:59.711 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.711+0000 7fb240d29640 1 -- 172.21.15.44:0/1488383725 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230031c40 con 0x7fb23c0cc6e0 2024-04-22T07:03:59.711 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.712+0000 7fb243f8f640 1 -- 172.21.15.44:0/1488383725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c0cc6e0 msgr2=0x7fb23c0cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.711 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.712+0000 7fb243f8f640 1 --2- 172.21.15.44:0/1488383725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c0cc6e0 0x7fb23c0cd230 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0cddb0 tx=0x7fb23002b910 comp rx=0 tx=0).stop 2024-04-22T07:03:59.711 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.712+0000 7fb243f8f640 1 -- 172.21.15.44:0/1488383725 shutdown_connections 2024-04-22T07:03:59.711 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.712+0000 7fb243f8f640 1 --2- 172.21.15.44:0/1488383725 >> v2:172.21.15.70:3300/0 conn(0x7fb23c0c5540 0x7fb23c0cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.711 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.712+0000 7fb243f8f640 1 --2- 172.21.15.44:0/1488383725 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c0cc6e0 0x7fb23c0cd230 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.711 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.712+0000 7fb243f8f640 1 -- 172.21.15.44:0/1488383725 >> 172.21.15.44:0/1488383725 conn(0x7fb23c062570 msgr2=0x7fb23c062980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:59.711 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.712+0000 7fb243f8f640 1 -- 172.21.15.44:0/1488383725 shutdown_connections 2024-04-22T07:03:59.711 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.712+0000 7fb243f8f640 1 -- 172.21.15.44:0/1488383725 wait complete. 2024-04-22T07:03:59.713 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.713+0000 7fb243f8f640 1 Processor -- start 2024-04-22T07:03:59.713 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.713+0000 7fb243f8f640 1 -- start start 2024-04-22T07:03:59.714 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb243f8f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb23c0c5540 0x7fb23c0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.714 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb243f8f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb23c0cc6e0 0x7fb23c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.714 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb243f8f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c1550b0 0x7fb23c0c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.714 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb243f8f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c0cb7a0 con 0x7fb23c1550b0 2024-04-22T07:03:59.714 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb243f8f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c15afb0 con 0x7fb23c0cc6e0 2024-04-22T07:03:59.715 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb243f8f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb23c15ebc0 con 0x7fb23c0c5540 2024-04-22T07:03:59.715 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb24252c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c1550b0 0x7fb23c0c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.715 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb24252c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c1550b0 0x7fb23c0c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55620/0 (socket says 172.21.15.44:55620) 2024-04-22T07:03:59.715 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb24252c640 1 -- 172.21.15.44:0/597045195 learned_addr learned my addr 172.21.15.44:0/597045195 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:03:59.716 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb24152a640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb23c0cc6e0 0x7fb23c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.716 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb241d2b640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb23c0c5540 0x7fb23c0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.717 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb24252c640 1 -- 172.21.15.44:0/597045195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb23c0c5540 msgr2=0x7fb23c0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.717 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb24252c640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb23c0c5540 0x7fb23c0cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.717 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb24252c640 1 -- 172.21.15.44:0/597045195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb23c0cc6e0 msgr2=0x7fb23c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.718 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb24252c640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb23c0cc6e0 0x7fb23c0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.718 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb24252c640 1 -- 172.21.15.44:0/597045195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c1f1010 con 0x7fb23c1550b0 2024-04-22T07:03:59.718 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.714+0000 7fb24152a640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb23c0cc6e0 0x7fb23c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:03:59.718 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.715+0000 7fb24252c640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c1550b0 0x7fb23c0c7e40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb23400e7f0 tx=0x7fb23400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:59.718 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.715+0000 7fb23a7fc640 1 -- 172.21.15.44:0/597045195 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234017070 con 0x7fb23c1550b0 2024-04-22T07:03:59.718 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.715+0000 7fb23a7fc640 1 -- 172.21.15.44:0/597045195 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23400ee40 con 0x7fb23c1550b0 2024-04-22T07:03:59.718 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.715+0000 7fb23a7fc640 1 -- 172.21.15.44:0/597045195 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23400cb80 con 0x7fb23c1550b0 2024-04-22T07:03:59.718 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.715+0000 7fb243f8f640 1 -- 172.21.15.44:0/597045195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c1f12b0 con 0x7fb23c1550b0 2024-04-22T07:03:59.718 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.715+0000 7fb243f8f640 1 -- 172.21.15.44:0/597045195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1f1870 con 0x7fb23c1550b0 2024-04-22T07:03:59.719 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.716+0000 7fb243f8f640 1 -- 172.21.15.44:0/597045195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23c04fce0 con 0x7fb23c1550b0 2024-04-22T07:03:59.719 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.716+0000 7fb23a7fc640 1 -- 172.21.15.44:0/597045195 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fb23401a070 con 0x7fb23c1550b0 2024-04-22T07:03:59.723 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.716+0000 7fb23a7fc640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb22806c860 0x7fb22806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:03:59.723 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.717+0000 7fb23a7fc640 1 -- 172.21.15.44:0/597045195 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7fb23408ad20 con 0x7fb23c1550b0 2024-04-22T07:03:59.723 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.717+0000 7fb241d2b640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb22806c860 0x7fb22806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:03:59.723 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.717+0000 7fb241d2b640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb22806c860 0x7fb22806ed20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb22c007fd0 tx=0x7fb22c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:03:59.726 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.721+0000 7fb23a7fc640 1 -- 172.21.15.44:0/597045195 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb234059e20 con 0x7fb23c1550b0 2024-04-22T07:03:59.907 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.908+0000 7fb243f8f640 1 -- 172.21.15.44:0/597045195 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb23c153a60 con 0x7fb23c1550b0 2024-04-22T07:03:59.907 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.909+0000 7fb23a7fc640 1 -- 172.21.15.44:0/597045195 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7578 (secure 0 0 0) 0x7fb234059720 con 0x7fb23c1550b0 2024-04-22T07:03:59.908 INFO:teuthology.misc.health.smithi044.stdout: 2024-04-22T07:03:59.908 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":14,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:03:58.739044+0000","last_up_change":"2024-04-22T07:03:57.720998+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:03:59.909 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 -- 172.21.15.44:0/597045195 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb22806c860 msgr2=0x7fb22806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.909 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb22806c860 0x7fb22806ed20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb22c007fd0 tx=0x7fb22c009040 comp rx=0 tx=0).stop 2024-04-22T07:03:59.909 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 -- 172.21.15.44:0/597045195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c1550b0 msgr2=0x7fb23c0c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:03:59.909 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c1550b0 0x7fb23c0c7e40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb23400e7f0 tx=0x7fb23400ebf0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.909 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 -- 172.21.15.44:0/597045195 shutdown_connections 2024-04-22T07:03:59.910 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb22806c860 0x7fb22806ed20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.910 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb23c1550b0 0x7fb23c0c7e40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.910 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb23c0cc6e0 0x7fb23c0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.910 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 --2- 172.21.15.44:0/597045195 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb23c0c5540 0x7fb23c0cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:03:59.910 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 -- 172.21.15.44:0/597045195 >> 172.21.15.44:0/597045195 conn(0x7fb23c062570 msgr2=0x7fb23c062950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:03:59.910 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 -- 172.21.15.44:0/597045195 shutdown_connections 2024-04-22T07:03:59.910 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:03:59.911+0000 7fb243f8f640 1 -- 172.21.15.44:0/597045195 wait complete. 2024-04-22T07:03:59.920 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-22T07:04:06.727 DEBUG:teuthology.orchestra.run.smithi044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:06.831 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.833+0000 7fc1e7626640 1 Processor -- start 2024-04-22T07:04:06.831 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.834+0000 7fc1e7626640 1 -- start start 2024-04-22T07:04:06.833 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.834+0000 7fc1e7626640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1e0152aa0 0x7fc1e01593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.833 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.834+0000 7fc1e7626640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e00c9130 con 0x7fc1e00c3440 2024-04-22T07:04:06.833 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.834+0000 7fc1e7626640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e015c6c0 con 0x7fc1e015bee0 2024-04-22T07:04:06.833 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.834+0000 7fc1e7626640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc1e01598f0 con 0x7fc1e0152aa0 2024-04-22T07:04:06.833 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.835+0000 7fc1e4bc1640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fc1e00c3440 0x7fc1e00cabc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37906/0 (socket says 172.21.15.44:37906) 2024-04-22T07:04:06.833 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.835+0000 7fc1e4bc1640 1 -- 172.21.15.44:0/1507782630 learned_addr learned my addr 172.21.15.44:0/1507782630 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:06.833 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.835+0000 7fc1e53c2640 1 --2- 172.21.15.44:0/1507782630 >> v2:172.21.15.70:3300/0 conn(0x7fc1e0152aa0 0x7fc1e01593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.833 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.835+0000 7fc1d7fff640 1 -- 172.21.15.44:0/1507782630 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4268263828 0 0) 0x7fc1e00c9130 con 0x7fc1e00c3440 2024-04-22T07:04:06.834 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.835+0000 7fc1d7fff640 1 -- 172.21.15.44:0/1507782630 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1cc003520 con 0x7fc1e00c3440 2024-04-22T07:04:06.834 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.835+0000 7fc1d7fff640 1 -- 172.21.15.44:0/1507782630 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1639117154 0 0) 0x7fc1cc003520 con 0x7fc1e00c3440 2024-04-22T07:04:06.834 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.835+0000 7fc1e53c2640 1 -- 172.21.15.44:0/1507782630 >> v1:172.21.15.70:6789/0 conn(0x7fc1e015bee0 legacy=0x7fc1e0071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.834 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.835+0000 7fc1e53c2640 1 -- 172.21.15.44:0/1507782630 >> v1:172.21.15.44:6789/0 conn(0x7fc1e00c3440 legacy=0x7fc1e00cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.834 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.835+0000 7fc1e53c2640 1 -- 172.21.15.44:0/1507782630 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e01588f0 con 0x7fc1e0152aa0 2024-04-22T07:04:06.835 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.836+0000 7fc1e53c2640 1 --2- 172.21.15.44:0/1507782630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e0152aa0 0x7fc1e01593b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc0097a0 tx=0x7fc1dc02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=426732dc80dcd3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:06.835 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.836+0000 7fc1d7fff640 1 -- 172.21.15.44:0/1507782630 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc037070 con 0x7fc1e0152aa0 2024-04-22T07:04:06.835 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.836+0000 7fc1d7fff640 1 -- 172.21.15.44:0/1507782630 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1dc030510 con 0x7fc1e0152aa0 2024-04-22T07:04:06.835 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.836+0000 7fc1d7fff640 1 -- 172.21.15.44:0/1507782630 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0001670 con 0x7fc1e0152aa0 2024-04-22T07:04:06.835 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.837+0000 7fc1e7626640 1 -- 172.21.15.44:0/1507782630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e0152aa0 msgr2=0x7fc1e01593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.835 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.837+0000 7fc1e7626640 1 --2- 172.21.15.44:0/1507782630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e0152aa0 0x7fc1e01593b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc0097a0 tx=0x7fc1dc02b790 comp rx=0 tx=0).stop 2024-04-22T07:04:06.835 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.837+0000 7fc1e7626640 1 -- 172.21.15.44:0/1507782630 shutdown_connections 2024-04-22T07:04:06.836 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.837+0000 7fc1e7626640 1 --2- 172.21.15.44:0/1507782630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e0152aa0 0x7fc1e01593b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.836 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.837+0000 7fc1e7626640 1 -- 172.21.15.44:0/1507782630 >> 172.21.15.44:0/1507782630 conn(0x7fc1e006ece0 msgr2=0x7fc1e006f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:06.836 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.837+0000 7fc1e7626640 1 -- 172.21.15.44:0/1507782630 shutdown_connections 2024-04-22T07:04:06.836 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.837+0000 7fc1e7626640 1 -- 172.21.15.44:0/1507782630 wait complete. 2024-04-22T07:04:06.838 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.838+0000 7fc1e7626640 1 Processor -- start 2024-04-22T07:04:06.838 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.838+0000 7fc1e7626640 1 -- start start 2024-04-22T07:04:06.838 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e7626640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e00c3440 0x7fc1e00c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.838 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e7626640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc1e0152aa0 0x7fc1e00c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.838 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e7626640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e015bee0 0x7fc1e015acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.838 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e7626640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e0158bb0 con 0x7fc1e00c3440 2024-04-22T07:04:06.838 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e7626640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e015cc60 con 0x7fc1e015bee0 2024-04-22T07:04:06.838 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e7626640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e0158e90 con 0x7fc1e0152aa0 2024-04-22T07:04:06.838 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e53c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e00c3440 0x7fc1e00c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.838 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e53c2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e00c3440 0x7fc1e00c38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39718/0 (socket says 172.21.15.44:39718) 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e53c2640 1 -- 172.21.15.44:0/923447884 learned_addr learned my addr 172.21.15.44:0/923447884 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e4bc1640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc1e0152aa0 0x7fc1e00c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e5bc3640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e015bee0 0x7fc1e015acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e53c2640 1 -- 172.21.15.44:0/923447884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc1e0152aa0 msgr2=0x7fc1e00c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e53c2640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc1e0152aa0 0x7fc1e00c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e53c2640 1 -- 172.21.15.44:0/923447884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e015bee0 msgr2=0x7fc1e015acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e53c2640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e015bee0 0x7fc1e015acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e53c2640 1 -- 172.21.15.44:0/923447884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e0148ff0 con 0x7fc1e00c3440 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.839+0000 7fc1e53c2640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e00c3440 0x7fc1e00c38e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc000c00 tx=0x7fc1dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.840+0000 7fc1d5ffb640 1 -- 172.21.15.44:0/923447884 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc042070 con 0x7fc1e00c3440 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.840+0000 7fc1d5ffb640 1 -- 172.21.15.44:0/923447884 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1dc03e870 con 0x7fc1e00c3440 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.840+0000 7fc1d5ffb640 1 -- 172.21.15.44:0/923447884 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc037040 con 0x7fc1e00c3440 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.840+0000 7fc1e7626640 1 -- 172.21.15.44:0/923447884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e01492b0 con 0x7fc1e00c3440 2024-04-22T07:04:06.839 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.840+0000 7fc1e7626640 1 -- 172.21.15.44:0/923447884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e01eecb0 con 0x7fc1e00c3440 2024-04-22T07:04:06.840 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.841+0000 7fc1e7626640 1 -- 172.21.15.44:0/923447884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e0068af0 con 0x7fc1e00c3440 2024-04-22T07:04:06.840 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.841+0000 7fc1d5ffb640 1 -- 172.21.15.44:0/923447884 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1dc045070 con 0x7fc1e00c3440 2024-04-22T07:04:06.840 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.842+0000 7fc1d5ffb640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc1c406c920 0x7fc1c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:06.840 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.842+0000 7fc1d5ffb640 1 -- 172.21.15.44:0/923447884 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fc1dc0ad1a0 con 0x7fc1e00c3440 2024-04-22T07:04:06.840 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.842+0000 7fc1e4bc1640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc1c406c920 0x7fc1c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:06.841 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.842+0000 7fc1e4bc1640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc1c406c920 0x7fc1c406ede0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc1e01ec4f0 tx=0x7fc1d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:06.844 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:06.847+0000 7fc1d5ffb640 1 -- 172.21.15.44:0/923447884 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc1e0068af0 con 0x7fc1e00c3440 2024-04-22T07:04:07.030 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.033+0000 7fc1e7626640 1 -- 172.21.15.44:0/923447884 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1e00cb310 con 0x7fc1e00c3440 2024-04-22T07:04:07.031 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.034+0000 7fc1d5ffb640 1 -- 172.21.15.44:0/923447884 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12836 (secure 0 0 0) 0x7fc1e00cb310 con 0x7fc1e00c3440 2024-04-22T07:04:07.032 INFO:teuthology.misc.health.smithi044.stdout: 2024-04-22T07:04:07.032 INFO:teuthology.misc.health.smithi044.stdout:{"epoch":19,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:04.473823+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:07.034 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 -- 172.21.15.44:0/923447884 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc1c406c920 msgr2=0x7fc1c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.034 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc1c406c920 0x7fc1c406ede0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc1e01ec4f0 tx=0x7fc1d0009040 comp rx=0 tx=0).stop 2024-04-22T07:04:07.034 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 -- 172.21.15.44:0/923447884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e00c3440 msgr2=0x7fc1e00c38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.034 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e00c3440 0x7fc1e00c38e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc000c00 tx=0x7fc1dc0046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.034 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 -- 172.21.15.44:0/923447884 shutdown_connections 2024-04-22T07:04:07.034 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc1c406c920 0x7fc1c406ede0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.034 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc1e0152aa0 0x7fc1e00c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.034 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1e015bee0 0x7fc1e015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.034 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 --2- 172.21.15.44:0/923447884 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc1e00c3440 0x7fc1e00c38e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.035 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 -- 172.21.15.44:0/923447884 >> 172.21.15.44:0/923447884 conn(0x7fc1e006ece0 msgr2=0x7fc1e0156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:07.035 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 -- 172.21.15.44:0/923447884 shutdown_connections 2024-04-22T07:04:07.035 INFO:teuthology.misc.health.smithi044.stderr:2024-04-22T07:04:07.036+0000 7fc1e7626640 1 -- 172.21.15.44:0/923447884 wait complete. 2024-04-22T07:04:07.043 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-22T07:04:07.043 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-22T07:04:07.044 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-22T07:04:07.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-22T07:04:07.046 INFO:tasks.ceph.mds.a:Started 2024-04-22T07:04:07.046 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-22T07:04:07.046 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-22T07:04:07.048 INFO:tasks.ceph.mds.c:Started 2024-04-22T07:04:07.049 INFO:tasks.ceph.mds.e:Restarting daemon 2024-04-22T07:04:07.049 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-04-22T07:04:07.051 INFO:tasks.ceph.mds.e:Started 2024-04-22T07:04:07.051 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-22T07:04:07.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-22T07:04:07.053 INFO:tasks.ceph.mds.b:Started 2024-04-22T07:04:07.054 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-22T07:04:07.054 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-22T07:04:07.056 INFO:tasks.ceph.mds.d:Started 2024-04-22T07:04:07.056 INFO:tasks.ceph.mds.f:Restarting daemon 2024-04-22T07:04:07.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-04-22T07:04:07.058 INFO:tasks.ceph.mds.f:Started 2024-04-22T07:04:07.059 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-22T07:04:07.059 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:07.124 INFO:tasks.ceph.mds.a.smithi044.stdout:starting mds.a at 2024-04-22T07:04:07.125 INFO:tasks.ceph.mds.c.smithi044.stdout:starting mds.c at 2024-04-22T07:04:07.126 INFO:tasks.ceph.mds.e.smithi044.stdout:starting mds.e at 2024-04-22T07:04:07.137 INFO:tasks.ceph.mds.b.smithi070.stdout:starting mds.b at 2024-04-22T07:04:07.138 INFO:tasks.ceph.mds.d.smithi070.stdout:starting mds.d at 2024-04-22T07:04:07.141 INFO:tasks.ceph.mds.f.smithi070.stdout:starting mds.f at 2024-04-22T07:04:07.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.169+0000 7f196025c640 1 Processor -- start 2024-04-22T07:04:07.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.169+0000 7f196025c640 1 -- start start 2024-04-22T07:04:07.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f196025c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1958154f70 0x7f19580d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f196025c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b4a70 0x7f19580d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f196025c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f195815b180 con 0x7f1958155350 2024-04-22T07:04:07.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f196025c640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f195815b460 con 0x7f1958154f70 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f196025c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f195815b740 con 0x7f19580b4a70 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f195dff8640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1958154f70 0x7f19580d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f195dff8640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1958154f70 0x7f19580d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39776/0 (socket says 172.21.15.44:39776) 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f195dff8640 1 -- 172.21.15.44:0/763799130 learned_addr learned my addr 172.21.15.44:0/763799130 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f195e7f9640 1 --2- 172.21.15.44:0/763799130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b4a70 0x7f19580d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f195dff8640 1 -- 172.21.15.44:0/763799130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b4a70 msgr2=0x7f19580d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f195dff8640 1 --2- 172.21.15.44:0/763799130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b4a70 0x7f19580d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f195dff8640 1 -- 172.21.15.44:0/763799130 >> v1:172.21.15.70:6789/0 conn(0x7f1958155350 legacy=0x7f19580b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.170+0000 7f195dff8640 1 -- 172.21.15.44:0/763799130 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195815ba20 con 0x7f1958154f70 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.171+0000 7f195dff8640 1 --2- 172.21.15.44:0/763799130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 0x7f19580d0900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f194c009870 tx=0x7f194c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838d0d453492776d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.171+0000 7f195cff6640 1 -- 172.21.15.44:0/763799130 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c034070 con 0x7f1958154f70 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.171+0000 7f195cff6640 1 -- 172.21.15.44:0/763799130 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194c030840 con 0x7f1958154f70 2024-04-22T07:04:07.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.171+0000 7f195cff6640 1 -- 172.21.15.44:0/763799130 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c030c10 con 0x7f1958154f70 2024-04-22T07:04:07.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.172+0000 7f196025c640 1 -- 172.21.15.44:0/763799130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 msgr2=0x7f19580d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.172+0000 7f196025c640 1 --2- 172.21.15.44:0/763799130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 0x7f19580d0900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f194c009870 tx=0x7f194c02b980 comp rx=0 tx=0).stop 2024-04-22T07:04:07.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.172+0000 7f196025c640 1 -- 172.21.15.44:0/763799130 shutdown_connections 2024-04-22T07:04:07.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.172+0000 7f196025c640 1 --2- 172.21.15.44:0/763799130 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b4a70 0x7f19580d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.172+0000 7f196025c640 1 --2- 172.21.15.44:0/763799130 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 0x7f19580d0900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.172+0000 7f196025c640 1 -- 172.21.15.44:0/763799130 >> 172.21.15.44:0/763799130 conn(0x7f19580735c0 msgr2=0x7f19580739d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:07.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.172+0000 7f196025c640 1 -- 172.21.15.44:0/763799130 shutdown_connections 2024-04-22T07:04:07.172 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.172+0000 7f196025c640 1 -- 172.21.15.44:0/763799130 wait complete. 2024-04-22T07:04:07.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.173+0000 7f196025c640 1 Processor -- start 2024-04-22T07:04:07.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.173+0000 7f196025c640 1 -- start start 2024-04-22T07:04:07.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f196025c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19580b4a70 0x7f19581f1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f196025c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 0x7f19580c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f196025c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b54d0 0x7f19580c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f196025c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f195815add0 con 0x7f1958154f70 2024-04-22T07:04:07.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f196025c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f19580caf30 con 0x7f19580b4a70 2024-04-22T07:04:07.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f196025c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f19581f1880 con 0x7f19580b54d0 2024-04-22T07:04:07.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f195d7f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 0x7f19580c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f195d7f7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 0x7f19580c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39792/0 (socket says 172.21.15.44:39792) 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f195d7f7640 1 -- 172.21.15.44:0/295196451 learned_addr learned my addr 172.21.15.44:0/295196451 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f195dff8640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19580b4a70 0x7f19581f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.174+0000 7f195e7f9640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b54d0 0x7f19580c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f195d7f7640 1 -- 172.21.15.44:0/295196451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b54d0 msgr2=0x7f19580c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f195d7f7640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b54d0 0x7f19580c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f195d7f7640 1 -- 172.21.15.44:0/295196451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19580b4a70 msgr2=0x7f19581f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f195d7f7640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19580b4a70 0x7f19581f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f195d7f7640 1 -- 172.21.15.44:0/295196451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19580c7ea0 con 0x7f1958154f70 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f195e7f9640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b54d0 0x7f19580c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f195d7f7640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 0x7f19580c5d60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f195400c430 tx=0x7f195400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f194a7fc640 1 -- 172.21.15.44:0/295196451 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540052d0 con 0x7f1958154f70 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f194a7fc640 1 -- 172.21.15.44:0/295196451 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1954005490 con 0x7f1958154f70 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f196025c640 1 -- 172.21.15.44:0/295196451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19581e6bd0 con 0x7f1958154f70 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f194a7fc640 1 -- 172.21.15.44:0/295196451 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954007260 con 0x7f1958154f70 2024-04-22T07:04:07.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.175+0000 7f196025c640 1 -- 172.21.15.44:0/295196451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19580c7b40 con 0x7f1958154f70 2024-04-22T07:04:07.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.176+0000 7f196025c640 1 -- 172.21.15.44:0/295196451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1958069ea0 con 0x7f1958154f70 2024-04-22T07:04:07.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.177+0000 7f194a7fc640 1 -- 172.21.15.44:0/295196451 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1954007400 con 0x7f1958154f70 2024-04-22T07:04:07.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.177+0000 7f194a7fc640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f193c06c920 0x7f193c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.177+0000 7f194a7fc640 1 -- 172.21.15.44:0/295196451 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f195408ad10 con 0x7f1958154f70 2024-04-22T07:04:07.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.177+0000 7f195dff8640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f193c06c920 0x7f193c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.178+0000 7f195dff8640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f193c06c920 0x7f193c06ede0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f194c036000 tx=0x7f194c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.182+0000 7f194a7fc640 1 -- 172.21.15.44:0/295196451 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1954058db0 con 0x7f1958154f70 2024-04-22T07:04:07.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.367+0000 7f196025c640 1 -- 172.21.15.44:0/295196451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19580cdea0 con 0x7f1958154f70 2024-04-22T07:04:07.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.367+0000 7f194a7fc640 1 -- 172.21.15.44:0/295196451 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12836 (secure 0 0 0) 0x7f195405d0b0 con 0x7f1958154f70 2024-04-22T07:04:07.365 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:07.365 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":19,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:04.473823+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:07.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 -- 172.21.15.44:0/295196451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f193c06c920 msgr2=0x7f193c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f193c06c920 0x7f193c06ede0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f194c036000 tx=0x7f194c03b040 comp rx=0 tx=0).stop 2024-04-22T07:04:07.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 -- 172.21.15.44:0/295196451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 msgr2=0x7f19580c5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 0x7f19580c5d60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f195400c430 tx=0x7f195400c8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 -- 172.21.15.44:0/295196451 shutdown_connections 2024-04-22T07:04:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f193c06c920 0x7f193c06ede0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19580b54d0 0x7f19580c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1958154f70 0x7f19580c5d60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 --2- 172.21.15.44:0/295196451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19580b4a70 0x7f19581f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 -- 172.21.15.44:0/295196451 >> 172.21.15.44:0/295196451 conn(0x7f19580735c0 msgr2=0x7f19580cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 -- 172.21.15.44:0/295196451 shutdown_connections 2024-04-22T07:04:07.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.369+0000 7f196025c640 1 -- 172.21.15.44:0/295196451 wait complete. 2024-04-22T07:04:07.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:04:01.723984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:07.378 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:07.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.483+0000 7f31fb7ce640 1 Processor -- start 2024-04-22T07:04:07.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.483+0000 7f31fb7ce640 1 -- start start 2024-04-22T07:04:07.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.484+0000 7f31fb7ce640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f31f40caec0 0x7f31f40b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.484+0000 7f31fb7ce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31f40cc920 0x7f31f40cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.484+0000 7f31fb7ce640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f41556a0 con 0x7f31f41552c0 2024-04-22T07:04:07.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.484+0000 7f31fb7ce640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f31f415b160 con 0x7f31f40caec0 2024-04-22T07:04:07.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.484+0000 7f31fb7ce640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f31f415b440 con 0x7f31f40cc920 2024-04-22T07:04:07.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.484+0000 7f31f8d69640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f31f40caec0 0x7f31f40b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.484+0000 7f31f8d69640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f31f40caec0 0x7f31f40b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39794/0 (socket says 172.21.15.44:39794) 2024-04-22T07:04:07.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.484+0000 7f31f956a640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f31f41552c0 0x7f31f40d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37926/0 (socket says 172.21.15.44:37926) 2024-04-22T07:04:07.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.484+0000 7f31f8d69640 1 -- 172.21.15.44:0/3123409929 learned_addr learned my addr 172.21.15.44:0/3123409929 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:07.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.484+0000 7f31f9d6b640 1 --2- 172.21.15.44:0/3123409929 >> v2:172.21.15.70:3300/0 conn(0x7f31f40cc920 0x7f31f40cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.485+0000 7f31f8d69640 1 -- 172.21.15.44:0/3123409929 >> v2:172.21.15.70:3300/0 conn(0x7f31f40cc920 msgr2=0x7f31f40cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:07.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.485+0000 7f31f8d69640 1 --2- 172.21.15.44:0/3123409929 >> v2:172.21.15.70:3300/0 conn(0x7f31f40cc920 0x7f31f40cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.485+0000 7f31f8d69640 1 -- 172.21.15.44:0/3123409929 >> v1:172.21.15.44:6789/0 conn(0x7f31f41552c0 legacy=0x7f31f40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.485+0000 7f31f8d69640 1 -- 172.21.15.44:0/3123409929 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f415b720 con 0x7f31f40caec0 2024-04-22T07:04:07.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.485+0000 7f31f8d69640 1 --2- 172.21.15.44:0/3123409929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40caec0 0x7f31f40b32a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f31f4073320 tx=0x7f31e402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848792f9af687c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.486+0000 7f31ebfff640 1 -- 172.21.15.44:0/3123409929 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4034070 con 0x7f31f40caec0 2024-04-22T07:04:07.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.486+0000 7f31ebfff640 1 -- 172.21.15.44:0/3123409929 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e4030860 con 0x7f31f40caec0 2024-04-22T07:04:07.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.486+0000 7f31ebfff640 1 -- 172.21.15.44:0/3123409929 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4030c30 con 0x7f31f40caec0 2024-04-22T07:04:07.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.487+0000 7f31fb7ce640 1 -- 172.21.15.44:0/3123409929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40caec0 msgr2=0x7f31f40b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.487+0000 7f31fb7ce640 1 --2- 172.21.15.44:0/3123409929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40caec0 0x7f31f40b32a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f31f4073320 tx=0x7f31e402b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.487+0000 7f31fb7ce640 1 -- 172.21.15.44:0/3123409929 shutdown_connections 2024-04-22T07:04:07.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.487+0000 7f31fb7ce640 1 --2- 172.21.15.44:0/3123409929 >> v2:172.21.15.70:3300/0 conn(0x7f31f40cc920 0x7f31f40cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.487+0000 7f31fb7ce640 1 --2- 172.21.15.44:0/3123409929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40caec0 0x7f31f40b32a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.487+0000 7f31fb7ce640 1 -- 172.21.15.44:0/3123409929 >> 172.21.15.44:0/3123409929 conn(0x7f31f40676b0 msgr2=0x7f31f4067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:07.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.487+0000 7f31fb7ce640 1 -- 172.21.15.44:0/3123409929 shutdown_connections 2024-04-22T07:04:07.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.487+0000 7f31fb7ce640 1 -- 172.21.15.44:0/3123409929 wait complete. 2024-04-22T07:04:07.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.488+0000 7f31fb7ce640 1 Processor -- start 2024-04-22T07:04:07.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.488+0000 7f31fb7ce640 1 -- start start 2024-04-22T07:04:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31fb7ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f40caec0 0x7f31f4157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31fb7ce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40cc920 0x7f31f41f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31fb7ce640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31f41552c0 0x7f31f41f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31fb7ce640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f31f40c6a00 con 0x7f31f40cc920 2024-04-22T07:04:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31fb7ce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f31f415bce0 con 0x7f31f40caec0 2024-04-22T07:04:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31fb7ce640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f31f40d1130 con 0x7f31f41552c0 2024-04-22T07:04:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f8d69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40cc920 0x7f31f41f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f8d69640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40cc920 0x7f31f41f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39800/0 (socket says 172.21.15.44:39800) 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f8d69640 1 -- 172.21.15.44:0/2550267940 learned_addr learned my addr 172.21.15.44:0/2550267940 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f956a640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f40caec0 0x7f31f4157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f9d6b640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31f41552c0 0x7f31f41f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f8d69640 1 -- 172.21.15.44:0/2550267940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31f41552c0 msgr2=0x7f31f41f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f8d69640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31f41552c0 0x7f31f41f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f8d69640 1 -- 172.21.15.44:0/2550267940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f40caec0 msgr2=0x7f31f4157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f8d69640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f40caec0 0x7f31f4157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f8d69640 1 -- 172.21.15.44:0/2550267940 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f4158280 con 0x7f31f40cc920 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31f8d69640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40cc920 0x7f31f41f1cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f31e4000c00 tx=0x7f31e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31e9ffb640 1 -- 172.21.15.44:0/2550267940 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4048070 con 0x7f31f40cc920 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31e9ffb640 1 -- 172.21.15.44:0/2550267940 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e4030660 con 0x7f31f40cc920 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.489+0000 7f31fb7ce640 1 -- 172.21.15.44:0/2550267940 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f4156d10 con 0x7f31f40cc920 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.490+0000 7f31e9ffb640 1 -- 172.21.15.44:0/2550267940 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4034040 con 0x7f31f40cc920 2024-04-22T07:04:07.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.490+0000 7f31fb7ce640 1 -- 172.21.15.44:0/2550267940 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f40b48d0 con 0x7f31f40cc920 2024-04-22T07:04:07.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.491+0000 7f31fb7ce640 1 -- 172.21.15.44:0/2550267940 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f4068b00 con 0x7f31f40cc920 2024-04-22T07:04:07.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.491+0000 7f31e9ffb640 1 -- 172.21.15.44:0/2550267940 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31e404b070 con 0x7f31f40cc920 2024-04-22T07:04:07.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.492+0000 7f31e9ffb640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f31d806c8f0 0x7f31d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.492+0000 7f31e9ffb640 1 -- 172.21.15.44:0/2550267940 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f31e40ad7a0 con 0x7f31f40cc920 2024-04-22T07:04:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.492+0000 7f31f956a640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f31d806c8f0 0x7f31d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.492+0000 7f31f956a640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f31d806c8f0 0x7f31d806edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31f0000f60 tx=0x7f31f0007e20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.496+0000 7f31e9ffb640 1 -- 172.21.15.44:0/2550267940 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31e407b8c0 con 0x7f31f40cc920 2024-04-22T07:04:07.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.683+0000 7f31fb7ce640 1 -- 172.21.15.44:0/2550267940 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f31f40ca970 con 0x7f31f40cc920 2024-04-22T07:04:07.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.684+0000 7f31e9ffb640 1 -- 172.21.15.44:0/2550267940 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f31e407fbc0 con 0x7f31f40cc920 2024-04-22T07:04:07.683 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T07:04:07.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 -- 172.21.15.44:0/2550267940 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f31d806c8f0 msgr2=0x7f31d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f31d806c8f0 0x7f31d806edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31f0000f60 tx=0x7f31f0007e20 comp rx=0 tx=0).stop 2024-04-22T07:04:07.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 -- 172.21.15.44:0/2550267940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40cc920 msgr2=0x7f31f41f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40cc920 0x7f31f41f1cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f31e4000c00 tx=0x7f31e4004730 comp rx=0 tx=0).stop 2024-04-22T07:04:07.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 -- 172.21.15.44:0/2550267940 shutdown_connections 2024-04-22T07:04:07.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f31d806c8f0 0x7f31d806edb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31f41552c0 0x7f31f41f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31f40cc920 0x7f31f41f1cd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 --2- 172.21.15.44:0/2550267940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f40caec0 0x7f31f4157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 -- 172.21.15.44:0/2550267940 >> 172.21.15.44:0/2550267940 conn(0x7f31f40676b0 msgr2=0x7f31f40cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:07.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 -- 172.21.15.44:0/2550267940 shutdown_connections 2024-04-22T07:04:07.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.686+0000 7f31fb7ce640 1 -- 172.21.15.44:0/2550267940 wait complete. 2024-04-22T07:04:07.696 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:07.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.798+0000 7f1ba3cc1640 1 Processor -- start 2024-04-22T07:04:07.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.798+0000 7f1ba3cc1640 1 -- start start 2024-04-22T07:04:07.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba3cc1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1b9c0cd6f0 0x7f1b9c15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba3cc1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1b9c0cb3e0 0x7f1b9c0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba3cc1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9c15ee90 con 0x7f1b9c154f70 2024-04-22T07:04:07.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba3cc1640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9c0cd260 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba3cc1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9c15f170 con 0x7f1b9c0cb3e0 2024-04-22T07:04:07.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba125c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1b9c0cd6f0 0x7f1b9c15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba125c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1b9c0cd6f0 0x7f1b9c15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39806/0 (socket says 172.21.15.44:39806) 2024-04-22T07:04:07.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba125c640 1 -- 172.21.15.44:0/3688336170 learned_addr learned my addr 172.21.15.44:0/3688336170 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:07.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba225e640 1 --2- 172.21.15.44:0/3688336170 >> v2:172.21.15.70:3300/0 conn(0x7f1b9c0cb3e0 0x7f1b9c0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba125c640 1 -- 172.21.15.44:0/3688336170 >> v2:172.21.15.70:3300/0 conn(0x7f1b9c0cb3e0 msgr2=0x7f1b9c0cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba125c640 1 --2- 172.21.15.44:0/3688336170 >> v2:172.21.15.70:3300/0 conn(0x7f1b9c0cb3e0 0x7f1b9c0cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba125c640 1 -- 172.21.15.44:0/3688336170 >> v1:172.21.15.70:6789/0 conn(0x7f1b9c154f70 legacy=0x7f1b9c0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba125c640 1 -- 172.21.15.44:0/3688336170 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c0cc930 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.799+0000 7f1ba225e640 1 --2- 172.21.15.44:0/3688336170 >> v2:172.21.15.70:3300/0 conn(0x7f1b9c0cb3e0 0x7f1b9c0cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.800+0000 7f1ba125c640 1 --2- 172.21.15.44:0/3688336170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 0x7f1b9c15ad30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0633d0 tx=0x7f1b8c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3948a0c0b9450cb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.800+0000 7f1ba0a5b640 1 -- 172.21.15.44:0/3688336170 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c034070 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.800+0000 7f1ba0a5b640 1 -- 172.21.15.44:0/3688336170 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c030860 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.800+0000 7f1ba0a5b640 1 -- 172.21.15.44:0/3688336170 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c030c30 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.800+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3688336170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 msgr2=0x7f1b9c15ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.800+0000 7f1ba3cc1640 1 --2- 172.21.15.44:0/3688336170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 0x7f1b9c15ad30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0633d0 tx=0x7f1b8c02b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.801+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3688336170 shutdown_connections 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.801+0000 7f1ba3cc1640 1 --2- 172.21.15.44:0/3688336170 >> v2:172.21.15.70:3300/0 conn(0x7f1b9c0cb3e0 0x7f1b9c0cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.801+0000 7f1ba3cc1640 1 --2- 172.21.15.44:0/3688336170 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 0x7f1b9c15ad30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.801+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3688336170 >> 172.21.15.44:0/3688336170 conn(0x7f1b9c067d40 msgr2=0x7f1b9c06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.801+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3688336170 shutdown_connections 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.801+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3688336170 wait complete. 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.802+0000 7f1ba3cc1640 1 Processor -- start 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.802+0000 7f1ba3cc1640 1 -- start start 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.802+0000 7f1ba3cc1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9c0cb3e0 0x7f1b9c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.802+0000 7f1ba3cc1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 0x7f1b9c157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.802+0000 7f1ba3cc1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b9c154f70 0x7f1b9c158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.802+0000 7f1ba3cc1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c155350 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.802+0000 7f1ba3cc1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9c0ccef0 con 0x7f1b9c154f70 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.802+0000 7f1ba3cc1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9c153a40 con 0x7f1b9c0cb3e0 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba125c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 0x7f1b9c157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba125c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 0x7f1b9c157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39820/0 (socket says 172.21.15.44:39820) 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba125c640 1 -- 172.21.15.44:0/3929098110 learned_addr learned my addr 172.21.15.44:0/3929098110 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba225e640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b9c154f70 0x7f1b9c158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba1a5d640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9c0cb3e0 0x7f1b9c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba125c640 1 -- 172.21.15.44:0/3929098110 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9c0cb3e0 msgr2=0x7f1b9c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba125c640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9c0cb3e0 0x7f1b9c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba125c640 1 -- 172.21.15.44:0/3929098110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b9c154f70 msgr2=0x7f1b9c158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba125c640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b9c154f70 0x7f1b9c158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba125c640 1 -- 172.21.15.44:0/3929098110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9c1522a0 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba125c640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 0x7f1b9c157ac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c02b9e0 tx=0x7f1b8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1b927fc640 1 -- 172.21.15.44:0/3929098110 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c048070 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1b927fc640 1 -- 172.21.15.44:0/3929098110 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c039d90 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3929098110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9c152580 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1b927fc640 1 -- 172.21.15.44:0/3929098110 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c034040 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.803+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3929098110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b9c1572c0 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.804+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3929098110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9c14eff0 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.805+0000 7f1b927fc640 1 -- 172.21.15.44:0/3929098110 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b8c04b070 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.805+0000 7f1b927fc640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b8006c920 0x7f1b8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:07.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.805+0000 7f1b927fc640 1 -- 172.21.15.44:0/3929098110 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f1b8c0adaa0 con 0x7f1b9c0cd6f0 2024-04-22T07:04:07.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.805+0000 7f1ba1a5d640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b8006c920 0x7f1b8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:07.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.806+0000 7f1ba1a5d640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b8006c920 0x7f1b8006ede0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b98006fd0 tx=0x7f1b98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:07.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:07.810+0000 7f1b927fc640 1 -- 172.21.15.44:0/3929098110 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b8c07c220 con 0x7f1b9c0cd6f0 2024-04-22T07:04:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.030+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3929098110 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b9c0cc500 con 0x7f1b9c0cd6f0 2024-04-22T07:04:08.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.031+0000 7f1b927fc640 1 -- 172.21.15.44:0/3929098110 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f1b8c07bb20 con 0x7f1b9c0cd6f0 2024-04-22T07:04:08.032 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:08.032 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-22T07:04:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 2 2024-04-22T07:04:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3929098110 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b8006c920 msgr2=0x7f1b8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b8006c920 0x7f1b8006ede0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b98006fd0 tx=0x7f1b98008040 comp rx=0 tx=0).stop 2024-04-22T07:04:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3929098110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 msgr2=0x7f1b9c157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 0x7f1b9c157ac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c02b9e0 tx=0x7f1b8c004730 comp rx=0 tx=0).stop 2024-04-22T07:04:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3929098110 shutdown_connections 2024-04-22T07:04:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b8006c920 0x7f1b8006ede0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b9c154f70 0x7f1b9c158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b9c0cd6f0 0x7f1b9c157ac0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 --2- 172.21.15.44:0/3929098110 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b9c0cb3e0 0x7f1b9c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3929098110 >> 172.21.15.44:0/3929098110 conn(0x7f1b9c067d40 msgr2=0x7f1b9c0ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3929098110 shutdown_connections 2024-04-22T07:04:08.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:08.033+0000 7f1ba3cc1640 1 -- 172.21.15.44:0/3929098110 wait complete. 2024-04-22T07:04:10.044 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:10.157 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.159+0000 7ff6b8575640 1 Processor -- start 2024-04-22T07:04:10.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.159+0000 7ff6b8575640 1 -- start start 2024-04-22T07:04:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b8575640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b0154f70 0x7ff6b015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b8575640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff6b00cb3e0 0x7ff6b00cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b8575640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6b015ee90 con 0x7ff6b00cd6f0 2024-04-22T07:04:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b8575640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff6b00cd260 con 0x7ff6b00cb3e0 2024-04-22T07:04:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b8575640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b015f170 con 0x7ff6b0154f70 2024-04-22T07:04:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b6b12640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff6b00cb3e0 0x7ff6b00cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.159 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b6311640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b0154f70 0x7ff6b015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b5b10640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff6b00cd6f0 0x7ff6b00c5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:38696/0 (socket says 172.21.15.44:38696) 2024-04-22T07:04:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b5b10640 1 -- 172.21.15.44:0/2472946886 learned_addr learned my addr 172.21.15.44:0/2472946886 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b6311640 1 -- 172.21.15.44:0/2472946886 >> v2:172.21.15.70:3300/0 conn(0x7ff6b00cb3e0 msgr2=0x7ff6b00cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b6311640 1 --2- 172.21.15.44:0/2472946886 >> v2:172.21.15.70:3300/0 conn(0x7ff6b00cb3e0 0x7ff6b00cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b6311640 1 -- 172.21.15.44:0/2472946886 >> v1:172.21.15.70:6789/0 conn(0x7ff6b00cd6f0 legacy=0x7ff6b00c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.160+0000 7ff6b6311640 1 -- 172.21.15.44:0/2472946886 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b00cc930 con 0x7ff6b0154f70 2024-04-22T07:04:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.161+0000 7ff6b6311640 1 --2- 172.21.15.44:0/2472946886 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b0154f70 0x7ff6b015ad30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0097a0 tx=0x7ff6ac02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec0dd1a64e4a5040 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.161+0000 7ff6b530f640 1 -- 172.21.15.44:0/2472946886 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac034070 con 0x7ff6b0154f70 2024-04-22T07:04:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.161+0000 7ff6b530f640 1 -- 172.21.15.44:0/2472946886 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ac030610 con 0x7ff6b0154f70 2024-04-22T07:04:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.161+0000 7ff6b530f640 1 -- 172.21.15.44:0/2472946886 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0309e0 con 0x7ff6b0154f70 2024-04-22T07:04:10.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.162+0000 7ff6b8575640 1 -- 172.21.15.44:0/2472946886 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b0154f70 msgr2=0x7ff6b015ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.162+0000 7ff6b8575640 1 --2- 172.21.15.44:0/2472946886 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b0154f70 0x7ff6b015ad30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0097a0 tx=0x7ff6ac02b960 comp rx=0 tx=0).stop 2024-04-22T07:04:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.162+0000 7ff6b8575640 1 -- 172.21.15.44:0/2472946886 shutdown_connections 2024-04-22T07:04:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.162+0000 7ff6b8575640 1 --2- 172.21.15.44:0/2472946886 >> v2:172.21.15.70:3300/0 conn(0x7ff6b00cb3e0 0x7ff6b00cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.162+0000 7ff6b8575640 1 --2- 172.21.15.44:0/2472946886 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b0154f70 0x7ff6b015ad30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.162+0000 7ff6b8575640 1 -- 172.21.15.44:0/2472946886 >> 172.21.15.44:0/2472946886 conn(0x7ff6b0067d40 msgr2=0x7ff6b006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.162+0000 7ff6b8575640 1 -- 172.21.15.44:0/2472946886 shutdown_connections 2024-04-22T07:04:10.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.162+0000 7ff6b8575640 1 -- 172.21.15.44:0/2472946886 wait complete. 2024-04-22T07:04:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.163+0000 7ff6b8575640 1 Processor -- start 2024-04-22T07:04:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.163+0000 7ff6b8575640 1 -- start start 2024-04-22T07:04:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b8575640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b00cb3e0 0x7ff6b01f0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b8575640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b00cd6f0 0x7ff6b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b8575640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b0154f70 0x7ff6b014bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b8575640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b0155350 con 0x7ff6b0154f70 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b8575640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff6b00ccef0 con 0x7ff6b00cd6f0 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b8575640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff6b0156280 con 0x7ff6b00cb3e0 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b6b12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b0154f70 0x7ff6b014bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b6b12640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b0154f70 0x7ff6b014bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39828/0 (socket says 172.21.15.44:39828) 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b6b12640 1 -- 172.21.15.44:0/3714584097 learned_addr learned my addr 172.21.15.44:0/3714584097 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b6311640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b00cb3e0 0x7ff6b01f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b5b10640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b00cd6f0 0x7ff6b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b6b12640 1 -- 172.21.15.44:0/3714584097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b00cb3e0 msgr2=0x7ff6b01f0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b6b12640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b00cb3e0 0x7ff6b01f0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b6b12640 1 -- 172.21.15.44:0/3714584097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b00cd6f0 msgr2=0x7ff6b00b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b6b12640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b00cd6f0 0x7ff6b00b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.164+0000 7ff6b6b12640 1 -- 172.21.15.44:0/3714584097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6b0155cc0 con 0x7ff6b0154f70 2024-04-22T07:04:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.165+0000 7ff6b6b12640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b0154f70 0x7ff6b014bc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800efc0 tx=0x7ff6a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.165+0000 7ff6a6ffd640 1 -- 172.21.15.44:0/3714584097 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8018020 con 0x7ff6b0154f70 2024-04-22T07:04:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.165+0000 7ff6a6ffd640 1 -- 172.21.15.44:0/3714584097 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6a8014480 con 0x7ff6b0154f70 2024-04-22T07:04:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.165+0000 7ff6b8575640 1 -- 172.21.15.44:0/3714584097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6b0155fa0 con 0x7ff6b0154f70 2024-04-22T07:04:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.165+0000 7ff6a6ffd640 1 -- 172.21.15.44:0/3714584097 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a8014870 con 0x7ff6b0154f70 2024-04-22T07:04:10.166 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.165+0000 7ff6b8575640 1 -- 172.21.15.44:0/3714584097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6b01f41e0 con 0x7ff6b0154f70 2024-04-22T07:04:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.166+0000 7ff6b8575640 1 -- 172.21.15.44:0/3714584097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6b00cc500 con 0x7ff6b0154f70 2024-04-22T07:04:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.166+0000 7ff6a6ffd640 1 -- 172.21.15.44:0/3714584097 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6a801b020 con 0x7ff6b0154f70 2024-04-22T07:04:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.167+0000 7ff6a6ffd640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff69406c8f0 0x7ff69406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.167+0000 7ff6a6ffd640 1 -- 172.21.15.44:0/3714584097 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7ff6a808c5b0 con 0x7ff6b0154f70 2024-04-22T07:04:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.167+0000 7ff6b6311640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff69406c8f0 0x7ff69406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.167+0000 7ff6b6311640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff69406c8f0 0x7ff69406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac036000 tx=0x7ff6ac03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.171+0000 7ff6a6ffd640 1 -- 172.21.15.44:0/3714584097 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff6a8022020 con 0x7ff6b0154f70 2024-04-22T07:04:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.394+0000 7ff6b8575640 1 -- 172.21.15.44:0/3714584097 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6b004fdc0 con 0x7ff6b0154f70 2024-04-22T07:04:10.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.395+0000 7ff6a6ffd640 1 -- 172.21.15.44:0/3714584097 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5512 (secure 0 0 0) 0x7ff6a808c070 con 0x7ff6b0154f70 2024-04-22T07:04:10.394 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:10.394 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4303,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4324,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T07:04:10.394 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 3 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 -- 172.21.15.44:0/3714584097 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff69406c8f0 msgr2=0x7ff69406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff69406c8f0 0x7ff69406edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac036000 tx=0x7ff6ac03b040 comp rx=0 tx=0).stop 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 -- 172.21.15.44:0/3714584097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b0154f70 msgr2=0x7ff6b014bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b0154f70 0x7ff6b014bc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800efc0 tx=0x7ff6a800c490 comp rx=0 tx=0).stop 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 -- 172.21.15.44:0/3714584097 shutdown_connections 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff69406c8f0 0x7ff69406edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff6b0154f70 0x7ff6b014bc90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6b00cd6f0 0x7ff6b00b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 --2- 172.21.15.44:0/3714584097 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff6b00cb3e0 0x7ff6b01f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 -- 172.21.15.44:0/3714584097 >> 172.21.15.44:0/3714584097 conn(0x7ff6b0067d40 msgr2=0x7ff6b0152c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 -- 172.21.15.44:0/3714584097 shutdown_connections 2024-04-22T07:04:10.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.397+0000 7ff6b8575640 1 -- 172.21.15.44:0/3714584097 wait complete. 2024-04-22T07:04:10.406 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:10.499 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:04:10.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:04:10.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.508+0000 7fcfc5c6d640 1 Processor -- start 2024-04-22T07:04:10.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.508+0000 7fcfc5c6d640 1 -- start start 2024-04-22T07:04:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfc5c6d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcfc0154f70 0x7fcfc015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfc5c6d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcfc00cd6f0 0x7fcfc00c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfc5c6d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc00c5d40 con 0x7fcfc00cb3e0 2024-04-22T07:04:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfc5c6d640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc00cd260 con 0x7fcfc0154f70 2024-04-22T07:04:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfc5c6d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc015ee90 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfbf7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcfc0154f70 0x7fcfc015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfbf7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcfc0154f70 0x7fcfc015ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39832/0 (socket says 172.21.15.44:39832) 2024-04-22T07:04:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfbf7fe640 1 -- 172.21.15.44:0/2429351664 learned_addr learned my addr 172.21.15.44:0/2429351664 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfbeffd640 1 --2- 172.21.15.44:0/2429351664 >> v2:172.21.15.70:3300/0 conn(0x7fcfc00cd6f0 0x7fcfc00c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.508 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfbf7fe640 1 -- 172.21.15.44:0/2429351664 >> v2:172.21.15.70:3300/0 conn(0x7fcfc00cd6f0 msgr2=0x7fcfc00c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfbf7fe640 1 --2- 172.21.15.44:0/2429351664 >> v2:172.21.15.70:3300/0 conn(0x7fcfc00cd6f0 0x7fcfc00c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfbf7fe640 1 -- 172.21.15.44:0/2429351664 >> v1:172.21.15.70:6789/0 conn(0x7fcfc00cb3e0 legacy=0x7fcfc015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.509 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.509+0000 7fcfbf7fe640 1 -- 172.21.15.44:0/2429351664 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc015f170 con 0x7fcfc0154f70 2024-04-22T07:04:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.510+0000 7fcfbf7fe640 1 --2- 172.21.15.44:0/2429351664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc0154f70 0x7fcfc015ad30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00c7890 tx=0x7fcfb002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55174c7d851851d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.510+0000 7fcfbe7fc640 1 -- 172.21.15.44:0/2429351664 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb002f850 con 0x7fcfc0154f70 2024-04-22T07:04:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.510+0000 7fcfbe7fc640 1 -- 172.21.15.44:0/2429351664 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb002f9f0 con 0x7fcfc0154f70 2024-04-22T07:04:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.510+0000 7fcfbe7fc640 1 -- 172.21.15.44:0/2429351664 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb002fdc0 con 0x7fcfc0154f70 2024-04-22T07:04:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.511+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/2429351664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc0154f70 msgr2=0x7fcfc015ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.511+0000 7fcfc5c6d640 1 --2- 172.21.15.44:0/2429351664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc0154f70 0x7fcfc015ad30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00c7890 tx=0x7fcfb002ba30 comp rx=0 tx=0).stop 2024-04-22T07:04:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.511+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/2429351664 shutdown_connections 2024-04-22T07:04:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.511+0000 7fcfc5c6d640 1 --2- 172.21.15.44:0/2429351664 >> v2:172.21.15.70:3300/0 conn(0x7fcfc00cd6f0 0x7fcfc00c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.511+0000 7fcfc5c6d640 1 --2- 172.21.15.44:0/2429351664 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc0154f70 0x7fcfc015ad30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.510 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.511+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/2429351664 >> 172.21.15.44:0/2429351664 conn(0x7fcfc0067d40 msgr2=0x7fcfc006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.511+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/2429351664 shutdown_connections 2024-04-22T07:04:10.511 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.511+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/2429351664 wait complete. 2024-04-22T07:04:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.512+0000 7fcfc5c6d640 1 Processor -- start 2024-04-22T07:04:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.512+0000 7fcfc5c6d640 1 -- start start 2024-04-22T07:04:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfc5c6d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc00cb3e0 0x7fcfc01f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfc5c6d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc00cd6f0 0x7fcfc015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfc5c6d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc0154f70 0x7fcfc01592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfc5c6d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc00ccef0 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfc5c6d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc00ccc10 con 0x7fcfc00cb3e0 2024-04-22T07:04:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfc5c6d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc0155350 con 0x7fcfc0154f70 2024-04-22T07:04:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc00cd6f0 0x7fcfc015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.513 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbeffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc00cd6f0 0x7fcfc015cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39842/0 (socket says 172.21.15.44:39842) 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbeffd640 1 -- 172.21.15.44:0/3676652957 learned_addr learned my addr 172.21.15.44:0/3676652957 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbffff640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc0154f70 0x7fcfc01592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbf7fe640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc00cb3e0 0x7fcfc01f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbeffd640 1 -- 172.21.15.44:0/3676652957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc0154f70 msgr2=0x7fcfc01592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbeffd640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc0154f70 0x7fcfc01592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbeffd640 1 -- 172.21.15.44:0/3676652957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc00cb3e0 msgr2=0x7fcfc01f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbeffd640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc00cb3e0 0x7fcfc01f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbeffd640 1 -- 172.21.15.44:0/3676652957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc01f52a0 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.513+0000 7fcfbeffd640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc00cd6f0 0x7fcfc015cc60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400e9b0 tx=0x7fcfb400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.514+0000 7fcfa7fff640 1 -- 172.21.15.44:0/3676652957 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4017070 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.514+0000 7fcfa7fff640 1 -- 172.21.15.44:0/3676652957 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb400c9a0 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.514+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/3676652957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc01518a0 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.514 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.514+0000 7fcfa7fff640 1 -- 172.21.15.44:0/3676652957 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb400cd90 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.514+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/3676652957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc0151e60 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.515+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/3676652957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc015eba0 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.515 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.515+0000 7fcfa7fff640 1 -- 172.21.15.44:0/3676652957 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfb401a070 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.515+0000 7fcfa7fff640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcfac06c8f0 0x7fcfac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.516+0000 7fcfa7fff640 1 -- 172.21.15.44:0/3676652957 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fcfb408b760 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.516+0000 7fcfbf7fe640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcfac06c8f0 0x7fcfac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.516 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.516+0000 7fcfbf7fe640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcfac06c8f0 0x7fcfac06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0036000 tx=0x7fcfb0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.518 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.520+0000 7fcfa7fff640 1 -- 172.21.15.44:0/3676652957 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfb4059890 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.706+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/3676652957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfc00cc500 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.707+0000 7fcfa7fff640 1 -- 172.21.15.44:0/3676652957 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12836 (secure 0 0 0) 0x7fcfb405db90 con 0x7fcfc00cd6f0 2024-04-22T07:04:10.706 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:10.706 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":19,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:04.473823+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/3676652957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcfac06c8f0 msgr2=0x7fcfac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcfac06c8f0 0x7fcfac06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0036000 tx=0x7fcfb0008030 comp rx=0 tx=0).stop 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/3676652957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc00cd6f0 msgr2=0x7fcfc015cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc00cd6f0 0x7fcfc015cc60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400e9b0 tx=0x7fcfb400ee70 comp rx=0 tx=0).stop 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/3676652957 shutdown_connections 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcfac06c8f0 0x7fcfac06edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcfc0154f70 0x7fcfc01592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcfc00cd6f0 0x7fcfc015cc60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 --2- 172.21.15.44:0/3676652957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfc00cb3e0 0x7fcfc01f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/3676652957 >> 172.21.15.44:0/3676652957 conn(0x7fcfc0067d40 msgr2=0x7fcfc014f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/3676652957 shutdown_connections 2024-04-22T07:04:10.708 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.709+0000 7fcfc5c6d640 1 -- 172.21.15.44:0/3676652957 wait complete. 2024-04-22T07:04:10.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:04:01.723984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:10.719 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.824+0000 7fa2eca6d640 1 Processor -- start 2024-04-22T07:04:10.823 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.824+0000 7fa2eca6d640 1 -- start start 2024-04-22T07:04:10.826 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2eca6d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa2e80caec0 0x7fa2e80b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2eca6d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e80cc920 0x7fa2e80cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2eca6d640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e81556a0 con 0x7fa2e81552c0 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2eca6d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e815b160 con 0x7fa2e80caec0 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2eca6d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e815b440 con 0x7fa2e80cc920 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2e659c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa2e81552c0 0x7fa2e80d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37930/0 (socket says 172.21.15.44:37930) 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2e659c640 1 -- 172.21.15.44:0/1190462556 learned_addr learned my addr 172.21.15.44:0/1190462556 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4042487241 0 0) 0x7fa2e81556a0 con 0x7fa2e81552c0 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2d4003520 con 0x7fa2e81552c0 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2e6d9d640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e80cc920 0x7fa2e80cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.825+0000 7fa2e5d9b640 1 --2- 172.21.15.44:0/1190462556 >> v2:172.21.15.70:3300/0 conn(0x7fa2e80caec0 0x7fa2e80b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4107954705 0 0) 0x7fa2d4003520 con 0x7fa2e81552c0 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e80cc920 msgr2=0x7fa2e80cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.827 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e80cc920 0x7fa2e80cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 >> v2:172.21.15.70:3300/0 conn(0x7fa2e80caec0 msgr2=0x7fa2e80b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 --2- 172.21.15.44:0/1190462556 >> v2:172.21.15.70:3300/0 conn(0x7fa2e80caec0 0x7fa2e80b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e815b720 con 0x7fa2e81552c0 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3718881788 0 0) 0x7fa2e0001670 con 0x7fa2e81552c0 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 >> v1:172.21.15.44:6789/0 conn(0x7fa2e81552c0 legacy=0x7fa2e80d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 0x7fa2d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2d4004800 0x7fa2d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e80cc920 0x7fa2d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e815b160 con 0x7fa2d40039b0 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e815b440 con 0x7fa2d4004800 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d400b9c0 con 0x7fa2e80cc920 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e659c640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 0x7fa2d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e5d9b640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2d4004800 0x7fa2d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e6d9d640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e80cc920 0x7fa2d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.828 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e659c640 1 -- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e80cc920 msgr2=0x7fa2d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e659c640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e80cc920 0x7fa2d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e659c640 1 -- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2d4004800 msgr2=0x7fa2d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e659c640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2d4004800 0x7fa2d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e659c640 1 -- 172.21.15.44:0/1190462556 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d4003520 con 0x7fa2d40039b0 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.826+0000 7fa2e5d9b640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2d4004800 0x7fa2d4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.827+0000 7fa2e659c640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 0x7fa2d4003dd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0001050 tx=0x7fa2d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b9e87c3db7b323 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.827+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0007d70 con 0x7fa2d40039b0 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.827+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e0030070 con 0x7fa2d40039b0 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.827+0000 7fa2e559a640 1 -- 172.21.15.44:0/1190462556 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e002cd80 con 0x7fa2d40039b0 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.828+0000 7fa2eca6d640 1 -- 172.21.15.44:0/1190462556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 msgr2=0x7fa2d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.828+0000 7fa2eca6d640 1 --2- 172.21.15.44:0/1190462556 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 0x7fa2d4003dd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0001050 tx=0x7fa2d4005830 comp rx=0 tx=0).stop 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.828+0000 7fa2e6d9d640 1 -- 172.21.15.44:0/1190462556 reap_dead start 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.828+0000 7fa2eca6d640 1 -- 172.21.15.44:0/1190462556 shutdown_connections 2024-04-22T07:04:10.829 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.828+0000 7fa2eca6d640 1 -- 172.21.15.44:0/1190462556 >> 172.21.15.44:0/1190462556 conn(0x7fa2e80676b0 msgr2=0x7fa2e8067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.828+0000 7fa2eca6d640 1 -- 172.21.15.44:0/1190462556 shutdown_connections 2024-04-22T07:04:10.830 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.828+0000 7fa2eca6d640 1 -- 172.21.15.44:0/1190462556 wait complete. 2024-04-22T07:04:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.829+0000 7fa2eca6d640 1 Processor -- start 2024-04-22T07:04:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.829+0000 7fa2eca6d640 1 -- start start 2024-04-22T07:04:10.832 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2eca6d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 0x7fa2e814c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2eca6d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2e8158640 0x7fa2e814cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2eca6d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e815a640 0x7fa2e8155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2eca6d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e80c6a00 con 0x7fa2d40039b0 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2eca6d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e815bce0 con 0x7fa2e8158640 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2eca6d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e80d1130 con 0x7fa2e815a640 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 0x7fa2e814c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 0x7fa2e814c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39862/0 (socket says 172.21.15.44:39862) 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e659c640 1 -- 172.21.15.44:0/4076494200 learned_addr learned my addr 172.21.15.44:0/4076494200 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e5d9b640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2e8158640 0x7fa2e814cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e6d9d640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e815a640 0x7fa2e8155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e659c640 1 -- 172.21.15.44:0/4076494200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e815a640 msgr2=0x7fa2e8155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e659c640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e815a640 0x7fa2e8155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e659c640 1 -- 172.21.15.44:0/4076494200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2e8158640 msgr2=0x7fa2e814cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e659c640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2e8158640 0x7fa2e814cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:10.833 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e659c640 1 -- 172.21.15.44:0/4076494200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e814bb60 con 0x7fa2d40039b0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e6d9d640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e815a640 0x7fa2e8155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2e659c640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 0x7fa2e814c8b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000c300 tx=0x7fa2e0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2deffd640 1 -- 172.21.15.44:0/4076494200 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0004bf0 con 0x7fa2d40039b0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2deffd640 1 -- 172.21.15.44:0/4076494200 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e0004d90 con 0x7fa2d40039b0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2eca6d640 1 -- 172.21.15.44:0/4076494200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e8157df0 con 0x7fa2d40039b0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.830+0000 7fa2deffd640 1 -- 172.21.15.44:0/4076494200 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0007270 con 0x7fa2d40039b0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.831+0000 7fa2eca6d640 1 -- 172.21.15.44:0/4076494200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e814bdd0 con 0x7fa2d40039b0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.831+0000 7fa2eca6d640 1 -- 172.21.15.44:0/4076494200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e8068b00 con 0x7fa2d40039b0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.832+0000 7fa2deffd640 1 -- 172.21.15.44:0/4076494200 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2e0047020 con 0x7fa2d40039b0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.832+0000 7fa2deffd640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa2c806c8f0 0x7fa2c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.832+0000 7fa2deffd640 1 -- 172.21.15.44:0/4076494200 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fa2e00aca90 con 0x7fa2d40039b0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.832+0000 7fa2e5d9b640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa2c806c8f0 0x7fa2c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:10.834 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.833+0000 7fa2e5d9b640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa2c806c8f0 0x7fa2c806edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80cbaf0 tx=0x7fa2d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:10.835 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:10.837+0000 7fa2deffd640 1 -- 172.21.15.44:0/4076494200 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2e007b210 con 0x7fa2d40039b0 2024-04-22T07:04:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.018+0000 7fa2eca6d640 1 -- 172.21.15.44:0/4076494200 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa2e81591c0 con 0x7fa2d40039b0 2024-04-22T07:04:11.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.019+0000 7fa2deffd640 1 -- 172.21.15.44:0/4076494200 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7fa2e007ab10 con 0x7fa2d40039b0 2024-04-22T07:04:11.018 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 -- 172.21.15.44:0/4076494200 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa2c806c8f0 msgr2=0x7fa2c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa2c806c8f0 0x7fa2c806edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80cbaf0 tx=0x7fa2d0008040 comp rx=0 tx=0).stop 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 -- 172.21.15.44:0/4076494200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 msgr2=0x7fa2e814c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 0x7fa2e814c8b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000c300 tx=0x7fa2e0000fe0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 -- 172.21.15.44:0/4076494200 shutdown_connections 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa2c806c8f0 0x7fa2c806edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa2e815a640 0x7fa2e8155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2e8158640 0x7fa2e814cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 --2- 172.21.15.44:0/4076494200 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa2d40039b0 0x7fa2e814c8b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 -- 172.21.15.44:0/4076494200 >> 172.21.15.44:0/4076494200 conn(0x7fa2e80676b0 msgr2=0x7fa2e80cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 -- 172.21.15.44:0/4076494200 shutdown_connections 2024-04-22T07:04:11.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.021+0000 7fa2eca6d640 1 -- 172.21.15.44:0/4076494200 wait complete. 2024-04-22T07:04:11.030 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-22T07:04:11.130 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.132+0000 7fe227775640 1 Processor -- start 2024-04-22T07:04:11.131 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.132+0000 7fe227775640 1 -- start start 2024-04-22T07:04:11.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe227775640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe220154f50 0x7fe22015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe227775640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe22015bc20 con 0x7fe2200cc880 2024-04-22T07:04:11.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe227775640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe22015f1b0 con 0x7fe2200ccf30 2024-04-22T07:04:11.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe227775640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe2200cb7c0 con 0x7fe220154f50 2024-04-22T07:04:11.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe224d10640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fe2200cc880 0x7fe2200c54c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37938/0 (socket says 172.21.15.44:37938) 2024-04-22T07:04:11.132 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe224d10640 1 -- 172.21.15.44:0/1311656647 learned_addr learned my addr 172.21.15.44:0/1311656647 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:11.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe225511640 1 --2- 172.21.15.44:0/1311656647 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe220154f50 0x7fe22015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe217fff640 1 -- 172.21.15.44:0/1311656647 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 218054311 0 0) 0x7fe210001110 con 0x7fe2200cc880 2024-04-22T07:04:11.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe217fff640 1 -- 172.21.15.44:0/1311656647 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe20c003520 con 0x7fe2200cc880 2024-04-22T07:04:11.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe217fff640 1 -- 172.21.15.44:0/1311656647 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299580610 0 0) 0x7fe22015f1b0 con 0x7fe2200ccf30 2024-04-22T07:04:11.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe217fff640 1 -- 172.21.15.44:0/1311656647 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe210001110 con 0x7fe2200ccf30 2024-04-22T07:04:11.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe217fff640 1 -- 172.21.15.44:0/1311656647 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1897291478 0 0) 0x7fe20c003520 con 0x7fe2200cc880 2024-04-22T07:04:11.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe225511640 1 -- 172.21.15.44:0/1311656647 >> v1:172.21.15.70:6789/0 conn(0x7fe2200ccf30 legacy=0x7fe2201655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe225511640 1 -- 172.21.15.44:0/1311656647 >> v1:172.21.15.44:6789/0 conn(0x7fe2200cc880 legacy=0x7fe2200c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.133 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.133+0000 7fe225511640 1 -- 172.21.15.44:0/1311656647 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22015e3f0 con 0x7fe220154f50 2024-04-22T07:04:11.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.134+0000 7fe225511640 1 --2- 172.21.15.44:0/1311656647 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe220154f50 0x7fe22015b6e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009800 tx=0x7fe21c02b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=852da9bf5d12225c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.134+0000 7fe217fff640 1 -- 172.21.15.44:0/1311656647 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210001840 con 0x7fe220154f50 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.134+0000 7fe217fff640 1 -- 172.21.15.44:0/1311656647 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21c037070 con 0x7fe220154f50 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.134+0000 7fe217fff640 1 -- 172.21.15.44:0/1311656647 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c031760 con 0x7fe220154f50 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.135+0000 7fe227775640 1 -- 172.21.15.44:0/1311656647 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe220154f50 msgr2=0x7fe22015b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.135+0000 7fe227775640 1 --2- 172.21.15.44:0/1311656647 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe220154f50 0x7fe22015b6e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009800 tx=0x7fe21c02b880 comp rx=0 tx=0).stop 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.135+0000 7fe227775640 1 -- 172.21.15.44:0/1311656647 shutdown_connections 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.135+0000 7fe227775640 1 --2- 172.21.15.44:0/1311656647 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe220154f50 0x7fe22015b6e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.135+0000 7fe227775640 1 -- 172.21.15.44:0/1311656647 >> 172.21.15.44:0/1311656647 conn(0x7fe220067d40 msgr2=0x7fe22006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.135+0000 7fe227775640 1 -- 172.21.15.44:0/1311656647 shutdown_connections 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.135+0000 7fe227775640 1 -- 172.21.15.44:0/1311656647 wait complete. 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.136+0000 7fe227775640 1 Processor -- start 2024-04-22T07:04:11.135 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.136+0000 7fe227775640 1 -- start start 2024-04-22T07:04:11.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe227775640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200cc880 0x7fe220157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe227775640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2200ccf30 0x7fe22015d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe227775640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe220154f50 0x7fe220142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe227775640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe22015ec90 con 0x7fe2200cc880 2024-04-22T07:04:11.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe227775640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe22015e9b0 con 0x7fe220154f50 2024-04-22T07:04:11.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe227775640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe220155330 con 0x7fe2200ccf30 2024-04-22T07:04:11.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe225511640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200cc880 0x7fe220157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe225511640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200cc880 0x7fe220157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39870/0 (socket says 172.21.15.44:39870) 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe225511640 1 -- 172.21.15.44:0/1786835312 learned_addr learned my addr 172.21.15.44:0/1786835312 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe224d10640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2200ccf30 0x7fe22015d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe225d12640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe220154f50 0x7fe220142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe225511640 1 -- 172.21.15.44:0/1786835312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2200ccf30 msgr2=0x7fe22015d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe225511640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2200ccf30 0x7fe22015d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe225511640 1 -- 172.21.15.44:0/1786835312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe220154f50 msgr2=0x7fe220142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe225511640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe220154f50 0x7fe220142290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe225511640 1 -- 172.21.15.44:0/1786835312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2201f4610 con 0x7fe2200cc880 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe225511640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200cc880 0x7fe220157e00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe21c000c00 tx=0x7fe21c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.137+0000 7fe215ffb640 1 -- 172.21.15.44:0/1786835312 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c03b3f0 con 0x7fe2200cc880 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.138+0000 7fe215ffb640 1 -- 172.21.15.44:0/1786835312 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe21c03b590 con 0x7fe2200cc880 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.138+0000 7fe227775640 1 -- 172.21.15.44:0/1786835312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe220158580 con 0x7fe2200cc880 2024-04-22T07:04:11.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.138+0000 7fe215ffb640 1 -- 172.21.15.44:0/1786835312 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c03b940 con 0x7fe2200cc880 2024-04-22T07:04:11.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.138+0000 7fe227775640 1 -- 172.21.15.44:0/1786835312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe220156490 con 0x7fe2200cc880 2024-04-22T07:04:11.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.139+0000 7fe227775640 1 -- 172.21.15.44:0/1786835312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22015e1d0 con 0x7fe2200cc880 2024-04-22T07:04:11.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.139+0000 7fe215ffb640 1 -- 172.21.15.44:0/1786835312 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe21c03bae0 con 0x7fe2200cc880 2024-04-22T07:04:11.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.140+0000 7fe215ffb640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe20406c920 0x7fe20406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.140+0000 7fe215ffb640 1 -- 172.21.15.44:0/1786835312 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fe21c0acf60 con 0x7fe2200cc880 2024-04-22T07:04:11.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.140+0000 7fe224d10640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe20406c920 0x7fe20406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.140 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.140+0000 7fe224d10640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe20406c920 0x7fe20406ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe22015b720 tx=0x7fe210001e50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:11.142 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.144+0000 7fe215ffb640 1 -- 172.21.15.44:0/1786835312 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe21c040070 con 0x7fe2200cc880 2024-04-22T07:04:11.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.329+0000 7fe227775640 1 -- 172.21.15.44:0/1786835312 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe22004fdc0 con 0x7fe2200cc880 2024-04-22T07:04:11.328 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.329+0000 7fe215ffb640 1 -- 172.21.15.44:0/1786835312 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7fe21c07b080 con 0x7fe2200cc880 2024-04-22T07:04:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:11.328 INFO:teuthology.orchestra.run.smithi044.stdout:[ 2024-04-22T07:04:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: { 2024-04-22T07:04:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: "poolnum": 1, 2024-04-22T07:04:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: "poolname": ".mgr" 2024-04-22T07:04:11.328 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:04:11.328 INFO:teuthology.orchestra.run.smithi044.stdout:] 2024-04-22T07:04:11.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 -- 172.21.15.44:0/1786835312 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe20406c920 msgr2=0x7fe20406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe20406c920 0x7fe20406ede0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe22015b720 tx=0x7fe210001e50 comp rx=0 tx=0).stop 2024-04-22T07:04:11.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 -- 172.21.15.44:0/1786835312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200cc880 msgr2=0x7fe220157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200cc880 0x7fe220157e00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe21c000c00 tx=0x7fe21c0046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 -- 172.21.15.44:0/1786835312 shutdown_connections 2024-04-22T07:04:11.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe20406c920 0x7fe20406ede0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2200ccf30 0x7fe22015d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe220154f50 0x7fe220142290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 --2- 172.21.15.44:0/1786835312 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2200cc880 0x7fe220157e00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 -- 172.21.15.44:0/1786835312 >> 172.21.15.44:0/1786835312 conn(0x7fe220067d40 msgr2=0x7fe22014afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 -- 172.21.15.44:0/1786835312 shutdown_connections 2024-04-22T07:04:11.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.331+0000 7fe227775640 1 -- 172.21.15.44:0/1786835312 wait complete. 2024-04-22T07:04:11.340 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-22T07:04:11.340 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-22T07:04:11.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.445+0000 7f317d304640 1 Processor -- start 2024-04-22T07:04:11.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.446+0000 7f317d304640 1 -- start start 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.446+0000 7f317d304640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31780cc7c0 0x7f317815b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.446+0000 7f317d304640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31780ccf60 0x7f31780cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.446+0000 7f317d304640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f317815f1f0 con 0x7f31781550d0 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.446+0000 7f317d304640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f31781554b0 con 0x7f31780cc7c0 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.446+0000 7f317d304640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f317815ed60 con 0x7f31780ccf60 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.446+0000 7f3176ffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f31781550d0 0x7f31780c5440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37948/0 (socket says 172.21.15.44:37948) 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.446+0000 7f3176ffd640 1 -- 172.21.15.44:0/3785550619 learned_addr learned my addr 172.21.15.44:0/3785550619 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.447+0000 7f31767fc640 1 --2- 172.21.15.44:0/3785550619 >> v2:172.21.15.70:3300/0 conn(0x7f31780cc7c0 0x7f317815b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.447+0000 7f31777fe640 1 --2- 172.21.15.44:0/3785550619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31780ccf60 0x7f31780cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.447+0000 7f3175ffb640 1 -- 172.21.15.44:0/3785550619 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 786306268 0 0) 0x7f317815f1f0 con 0x7f31781550d0 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.447+0000 7f3175ffb640 1 -- 172.21.15.44:0/3785550619 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3160003520 con 0x7f31781550d0 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.447+0000 7f31767fc640 1 -- 172.21.15.44:0/3785550619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31780ccf60 msgr2=0x7f31780cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:11.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.447+0000 7f31767fc640 1 --2- 172.21.15.44:0/3785550619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31780ccf60 0x7f31780cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.447+0000 7f31767fc640 1 -- 172.21.15.44:0/3785550619 >> v1:172.21.15.44:6789/0 conn(0x7f31781550d0 legacy=0x7f31780c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.447+0000 7f31767fc640 1 -- 172.21.15.44:0/3785550619 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31780cb740 con 0x7f31780cc7c0 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.448+0000 7f31767fc640 1 --2- 172.21.15.44:0/3785550619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31780cc7c0 0x7f317815b760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f316c0097a0 tx=0x7f316c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d20f0a38ea97876f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.448+0000 7f3175ffb640 1 -- 172.21.15.44:0/3785550619 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c035020 con 0x7f31780cc7c0 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.448+0000 7f3175ffb640 1 -- 172.21.15.44:0/3785550619 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c031830 con 0x7f31780cc7c0 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.448+0000 7f3175ffb640 1 -- 172.21.15.44:0/3785550619 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c031be0 con 0x7f31780cc7c0 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.448+0000 7f317d304640 1 -- 172.21.15.44:0/3785550619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31780cc7c0 msgr2=0x7f317815b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.448+0000 7f317d304640 1 --2- 172.21.15.44:0/3785550619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31780cc7c0 0x7f317815b760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f316c0097a0 tx=0x7f316c02b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.448+0000 7f317d304640 1 -- 172.21.15.44:0/3785550619 shutdown_connections 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.448+0000 7f317d304640 1 --2- 172.21.15.44:0/3785550619 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31780ccf60 0x7f31780cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.448+0000 7f317d304640 1 --2- 172.21.15.44:0/3785550619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31780cc7c0 0x7f317815b760 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.448+0000 7f317d304640 1 -- 172.21.15.44:0/3785550619 >> 172.21.15.44:0/3785550619 conn(0x7f317806b970 msgr2=0x7f317806bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.449+0000 7f317d304640 1 -- 172.21.15.44:0/3785550619 shutdown_connections 2024-04-22T07:04:11.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.449+0000 7f317d304640 1 -- 172.21.15.44:0/3785550619 wait complete. 2024-04-22T07:04:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.450+0000 7f317d304640 1 Processor -- start 2024-04-22T07:04:11.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.450+0000 7f317d304640 1 -- start start 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f317d304640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31780cc7c0 0x7f317815a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f317d304640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31780ccf60 0x7f317815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f317d304640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31781550d0 0x7f31781f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f317d304640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f317815e710 con 0x7f31780ccf60 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f317d304640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f317815bca0 con 0x7f31781550d0 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f317d304640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f317815e9f0 con 0x7f31780cc7c0 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f31767fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31780ccf60 0x7f317815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f31767fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31780ccf60 0x7f317815da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39882/0 (socket says 172.21.15.44:39882) 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f31767fc640 1 -- 172.21.15.44:0/1907580758 learned_addr learned my addr 172.21.15.44:0/1907580758 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f31777fe640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31781550d0 0x7f31781f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f3176ffd640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31780cc7c0 0x7f317815a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f31767fc640 1 -- 172.21.15.44:0/1907580758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31780cc7c0 msgr2=0x7f317815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f31767fc640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31780cc7c0 0x7f317815a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f31767fc640 1 -- 172.21.15.44:0/1907580758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31781550d0 msgr2=0x7f31781f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f31767fc640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31781550d0 0x7f31781f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f31767fc640 1 -- 172.21.15.44:0/1907580758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317815ab40 con 0x7f31780ccf60 2024-04-22T07:04:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f31767fc640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31780ccf60 0x7f317815da00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f316c02b910 tx=0x7f316c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f315f7fe640 1 -- 172.21.15.44:0/1907580758 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c040070 con 0x7f31780ccf60 2024-04-22T07:04:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f315f7fe640 1 -- 172.21.15.44:0/1907580758 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c03a910 con 0x7f31780ccf60 2024-04-22T07:04:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f317d304640 1 -- 172.21.15.44:0/1907580758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3178154150 con 0x7f31780ccf60 2024-04-22T07:04:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f315f7fe640 1 -- 172.21.15.44:0/1907580758 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c035040 con 0x7f31780ccf60 2024-04-22T07:04:11.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.451+0000 7f317d304640 1 -- 172.21.15.44:0/1907580758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317815b0e0 con 0x7f31780ccf60 2024-04-22T07:04:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.452+0000 7f317d304640 1 -- 172.21.15.44:0/1907580758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31781594d0 con 0x7f31780ccf60 2024-04-22T07:04:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.453+0000 7f315f7fe640 1 -- 172.21.15.44:0/1907580758 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f316c04b060 con 0x7f31780ccf60 2024-04-22T07:04:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.453+0000 7f315f7fe640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f316406c8f0 0x7f316406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.453+0000 7f315f7fe640 1 -- 172.21.15.44:0/1907580758 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f316c0ace30 con 0x7f31780ccf60 2024-04-22T07:04:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.453+0000 7f3176ffd640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f316406c8f0 0x7f316406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.454+0000 7f3176ffd640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f316406c8f0 0x7f316406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3168005880 tx=0x7f3168009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:11.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.458+0000 7f315f7fe640 1 -- 172.21.15.44:0/1907580758 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31781594d0 con 0x7f31780ccf60 2024-04-22T07:04:11.646 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.647+0000 7f317d304640 1 -- 172.21.15.44:0/1907580758 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3178068b00 con 0x7f31780ccf60 2024-04-22T07:04:11.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.764+0000 7f315f7fe640 1 -- 172.21.15.44:0/1907580758 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f316c07af60 con 0x7f31780ccf60 2024-04-22T07:04:11.763 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_metadata' created 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 -- 172.21.15.44:0/1907580758 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f316406c8f0 msgr2=0x7f316406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f316406c8f0 0x7f316406edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3168005880 tx=0x7f3168009040 comp rx=0 tx=0).stop 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 -- 172.21.15.44:0/1907580758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31780ccf60 msgr2=0x7f317815da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31780ccf60 0x7f317815da00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f316c02b910 tx=0x7f316c0046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 -- 172.21.15.44:0/1907580758 shutdown_connections 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f316406c8f0 0x7f316406edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31781550d0 0x7f31781f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f31780ccf60 0x7f317815da00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 --2- 172.21.15.44:0/1907580758 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f31780cc7c0 0x7f317815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 -- 172.21.15.44:0/1907580758 >> 172.21.15.44:0/1907580758 conn(0x7f317806b970 msgr2=0x7f31780d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 -- 172.21.15.44:0/1907580758 shutdown_connections 2024-04-22T07:04:11.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.767+0000 7f317d304640 1 -- 172.21.15.44:0/1907580758 wait complete. 2024-04-22T07:04:11.780 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-22T07:04:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.889+0000 7fcc9a99d640 1 Processor -- start 2024-04-22T07:04:11.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.889+0000 7fcc9a99d640 1 -- start start 2024-04-22T07:04:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.890+0000 7fcc9a99d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc94154f60 0x7fcc9406c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.890+0000 7fcc9a99d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcc9406be80 0x7fcc9406cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.890+0000 7fcc9a99d640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc9406c370 con 0x7fcc9406b970 2024-04-22T07:04:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.890+0000 7fcc9a99d640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fcc940c8730 con 0x7fcc9406be80 2024-04-22T07:04:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.890+0000 7fcc9a99d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcc940c8980 con 0x7fcc94154f60 2024-04-22T07:04:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.890+0000 7fcc937fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcc9406be80 0x7fcc9406cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.890+0000 7fcc98f3a640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fcc9406b970 0x7fcc9415adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37958/0 (socket says 172.21.15.44:37958) 2024-04-22T07:04:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.890+0000 7fcc937fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fcc9406be80 0x7fcc9406cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39888/0 (socket says 172.21.15.44:39888) 2024-04-22T07:04:11.890 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.890+0000 7fcc98f3a640 1 -- 172.21.15.44:0/2997607149 learned_addr learned my addr 172.21.15.44:0/2997607149 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.891+0000 7fcc93fff640 1 --2- 172.21.15.44:0/2997607149 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc94154f60 0x7fcc9406c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.891+0000 7fcc937fe640 1 -- 172.21.15.44:0/2997607149 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc94154f60 msgr2=0x7fcc9406c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.891+0000 7fcc937fe640 1 --2- 172.21.15.44:0/2997607149 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc94154f60 0x7fcc9406c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.891+0000 7fcc937fe640 1 -- 172.21.15.44:0/2997607149 >> v1:172.21.15.44:6789/0 conn(0x7fcc9406b970 legacy=0x7fcc9415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.891+0000 7fcc92ffd640 1 -- 172.21.15.44:0/2997607149 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3703587014 0 0) 0x7fcc9406c370 con 0x7fcc9406b970 2024-04-22T07:04:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.891+0000 7fcc937fe640 1 -- 172.21.15.44:0/2997607149 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc940c8c60 con 0x7fcc9406be80 2024-04-22T07:04:11.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.891+0000 7fcc93fff640 1 --2- 172.21.15.44:0/2997607149 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc94154f60 0x7fcc9406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc937fe640 1 --2- 172.21.15.44:0/2997607149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9406be80 0x7fcc9406cc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc880097a0 tx=0x7fcc8802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331ff1001f8f61bd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc92ffd640 1 -- 172.21.15.44:0/2997607149 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc880315a0 con 0x7fcc9406be80 2024-04-22T07:04:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc92ffd640 1 -- 172.21.15.44:0/2997607149 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc88037070 con 0x7fcc9406be80 2024-04-22T07:04:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc92ffd640 1 -- 172.21.15.44:0/2997607149 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88031960 con 0x7fcc9406be80 2024-04-22T07:04:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc9a99d640 1 -- 172.21.15.44:0/2997607149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9406be80 msgr2=0x7fcc9406cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc9a99d640 1 --2- 172.21.15.44:0/2997607149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9406be80 0x7fcc9406cc90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcc880097a0 tx=0x7fcc8802b840 comp rx=0 tx=0).stop 2024-04-22T07:04:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc9a99d640 1 -- 172.21.15.44:0/2997607149 shutdown_connections 2024-04-22T07:04:11.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc9a99d640 1 --2- 172.21.15.44:0/2997607149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc9406be80 0x7fcc9406cc90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc9a99d640 1 --2- 172.21.15.44:0/2997607149 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc94154f60 0x7fcc9406c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc9a99d640 1 -- 172.21.15.44:0/2997607149 >> 172.21.15.44:0/2997607149 conn(0x7fcc940cd950 msgr2=0x7fcc9406e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc9a99d640 1 -- 172.21.15.44:0/2997607149 shutdown_connections 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.892+0000 7fcc9a99d640 1 -- 172.21.15.44:0/2997607149 wait complete. 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.893+0000 7fcc9a99d640 1 Processor -- start 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.894+0000 7fcc9a99d640 1 -- start start 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.894+0000 7fcc9a99d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc9406b970 0x7fcc941592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.894+0000 7fcc9a99d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc9406be80 0x7fcc94150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.894+0000 7fcc9a99d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc94154f60 0x7fcc941f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.894+0000 7fcc9a99d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc94155340 con 0x7fcc94154f60 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.894+0000 7fcc9a99d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcc940c9220 con 0x7fcc9406be80 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.894+0000 7fcc9a99d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcc940681d0 con 0x7fcc9406b970 2024-04-22T07:04:11.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.894+0000 7fcc98f3a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc94154f60 0x7fcc941f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.894+0000 7fcc98f3a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc94154f60 0x7fcc941f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39894/0 (socket says 172.21.15.44:39894) 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.894+0000 7fcc98f3a640 1 -- 172.21.15.44:0/789122950 learned_addr learned my addr 172.21.15.44:0/789122950 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc93fff640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc9406b970 0x7fcc941592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc937fe640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc9406be80 0x7fcc94150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc98f3a640 1 -- 172.21.15.44:0/789122950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc9406b970 msgr2=0x7fcc941592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc98f3a640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc9406b970 0x7fcc941592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc98f3a640 1 -- 172.21.15.44:0/789122950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc9406be80 msgr2=0x7fcc94150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc98f3a640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc9406be80 0x7fcc94150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc98f3a640 1 -- 172.21.15.44:0/789122950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc94159b00 con 0x7fcc94154f60 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc937fe640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc9406be80 0x7fcc94150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc98f3a640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc94154f60 0x7fcc941f1f10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00c780 tx=0x7fcc8c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc90ff9640 1 -- 172.21.15.44:0/789122950 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c018070 con 0x7fcc94154f60 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc90ff9640 1 -- 172.21.15.44:0/789122950 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc8c00a640 con 0x7fcc94154f60 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc90ff9640 1 -- 172.21.15.44:0/789122950 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00aab0 con 0x7fcc94154f60 2024-04-22T07:04:11.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc9a99d640 1 -- 172.21.15.44:0/789122950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc94159de0 con 0x7fcc94154f60 2024-04-22T07:04:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.895+0000 7fcc9a99d640 1 -- 172.21.15.44:0/789122950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc9415a3a0 con 0x7fcc94154f60 2024-04-22T07:04:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.896+0000 7fcc9a99d640 1 -- 172.21.15.44:0/789122950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc9415aa60 con 0x7fcc94154f60 2024-04-22T07:04:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.897+0000 7fcc90ff9640 1 -- 172.21.15.44:0/789122950 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc8c00ac50 con 0x7fcc94154f60 2024-04-22T07:04:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.897+0000 7fcc90ff9640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc8006c920 0x7fcc8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.897+0000 7fcc90ff9640 1 -- 172.21.15.44:0/789122950 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7fcc8c08ade0 con 0x7fcc94154f60 2024-04-22T07:04:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.897+0000 7fcc93fff640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc8006c920 0x7fcc8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.897+0000 7fcc93fff640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc8006c920 0x7fcc8006ede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400a950 tx=0x7fcc84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:11.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:11.901+0000 7fcc90ff9640 1 -- 172.21.15.44:0/789122950 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc8c059480 con 0x7fcc94154f60 2024-04-22T07:04:12.092 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.094+0000 7fcc9a99d640 1 -- 172.21.15.44:0/789122950 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcc941f1cf0 con 0x7fcc94154f60 2024-04-22T07:04:12.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.767+0000 7fcc90ff9640 1 -- 172.21.15.44:0/789122950 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7fcc8c058d80 con 0x7fcc94154f60 2024-04-22T07:04:12.766 INFO:teuthology.orchestra.run.smithi044.stderr:pool 'cephfs_data' created 2024-04-22T07:04:12.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.770+0000 7fcc9a99d640 1 -- 172.21.15.44:0/789122950 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc8006c920 msgr2=0x7fcc8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.770+0000 7fcc9a99d640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc8006c920 0x7fcc8006ede0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400a950 tx=0x7fcc84008040 comp rx=0 tx=0).stop 2024-04-22T07:04:12.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.770+0000 7fcc9a99d640 1 -- 172.21.15.44:0/789122950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc94154f60 msgr2=0x7fcc941f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.770+0000 7fcc9a99d640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc94154f60 0x7fcc941f1f10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00c780 tx=0x7fcc8c00cc40 comp rx=0 tx=0).stop 2024-04-22T07:04:12.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.771+0000 7fcc9a99d640 1 -- 172.21.15.44:0/789122950 shutdown_connections 2024-04-22T07:04:12.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.771+0000 7fcc9a99d640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc8006c920 0x7fcc8006ede0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.771+0000 7fcc9a99d640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc94154f60 0x7fcc941f1f10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.771+0000 7fcc9a99d640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc9406be80 0x7fcc94150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.771+0000 7fcc9a99d640 1 --2- 172.21.15.44:0/789122950 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc9406b970 0x7fcc941592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.771+0000 7fcc9a99d640 1 -- 172.21.15.44:0/789122950 >> 172.21.15.44:0/789122950 conn(0x7fcc940cd950 msgr2=0x7fcc940676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.772+0000 7fcc9a99d640 1 -- 172.21.15.44:0/789122950 shutdown_connections 2024-04-22T07:04:12.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.772+0000 7fcc9a99d640 1 -- 172.21.15.44:0/789122950 wait complete. 2024-04-22T07:04:12.797 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-22T07:04:12.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.907+0000 7f1071b87640 1 Processor -- start 2024-04-22T07:04:12.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.907+0000 7f1071b87640 1 -- start start 2024-04-22T07:04:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f1071b87640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f106c0caec0 0x7f106c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f1071b87640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c0cc920 0x7f106c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f1071b87640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f106c1556a0 con 0x7f106c1552c0 2024-04-22T07:04:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f1071b87640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f106c15b160 con 0x7f106c0caec0 2024-04-22T07:04:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f1071b87640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f106c15b440 con 0x7f106c0cc920 2024-04-22T07:04:12.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f106affd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f106c0caec0 0x7f106c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f106affd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f106c0caec0 0x7f106c0b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39902/0 (socket says 172.21.15.44:39902) 2024-04-22T07:04:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f106affd640 1 -- 172.21.15.44:0/1914216702 learned_addr learned my addr 172.21.15.44:0/1914216702 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f106bfff640 1 --2- 172.21.15.44:0/1914216702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c0cc920 0x7f106c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f106affd640 1 -- 172.21.15.44:0/1914216702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c0cc920 msgr2=0x7f106c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f106affd640 1 --2- 172.21.15.44:0/1914216702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c0cc920 0x7f106c0cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f106affd640 1 -- 172.21.15.44:0/1914216702 >> v1:172.21.15.70:6789/0 conn(0x7f106c1552c0 legacy=0x7f106c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.908+0000 7f106affd640 1 -- 172.21.15.44:0/1914216702 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c15b720 con 0x7f106c0caec0 2024-04-22T07:04:12.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.909+0000 7f106affd640 1 --2- 172.21.15.44:0/1914216702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 0x7f106c0b32a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1060009870 tx=0x7f106002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a233074b7f7e71e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.909+0000 7f106a7fc640 1 -- 172.21.15.44:0/1914216702 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060034070 con 0x7f106c0caec0 2024-04-22T07:04:12.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.909+0000 7f106a7fc640 1 -- 172.21.15.44:0/1914216702 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1060030840 con 0x7f106c0caec0 2024-04-22T07:04:12.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.909+0000 7f106a7fc640 1 -- 172.21.15.44:0/1914216702 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1060030c10 con 0x7f106c0caec0 2024-04-22T07:04:12.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.910+0000 7f1071b87640 1 -- 172.21.15.44:0/1914216702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 msgr2=0x7f106c0b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.910+0000 7f1071b87640 1 --2- 172.21.15.44:0/1914216702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 0x7f106c0b32a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1060009870 tx=0x7f106002b980 comp rx=0 tx=0).stop 2024-04-22T07:04:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.910+0000 7f1071b87640 1 -- 172.21.15.44:0/1914216702 shutdown_connections 2024-04-22T07:04:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.910+0000 7f1071b87640 1 --2- 172.21.15.44:0/1914216702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c0cc920 0x7f106c0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.910+0000 7f1071b87640 1 --2- 172.21.15.44:0/1914216702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 0x7f106c0b32a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.910+0000 7f1071b87640 1 -- 172.21.15.44:0/1914216702 >> 172.21.15.44:0/1914216702 conn(0x7f106c0676b0 msgr2=0x7f106c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.910+0000 7f1071b87640 1 -- 172.21.15.44:0/1914216702 shutdown_connections 2024-04-22T07:04:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.910+0000 7f1071b87640 1 -- 172.21.15.44:0/1914216702 wait complete. 2024-04-22T07:04:12.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.911+0000 7f1071b87640 1 Processor -- start 2024-04-22T07:04:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.911+0000 7f1071b87640 1 -- start start 2024-04-22T07:04:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f1071b87640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 0x7f106c0cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f1071b87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f106c0cc920 0x7f106c1e7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f1071b87640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c1552c0 0x7f106c1f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f1071b87640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f106c0c6a00 con 0x7f106c0caec0 2024-04-22T07:04:12.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f1071b87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f106c15bce0 con 0x7f106c0cc920 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f1071b87640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f106c0d1130 con 0x7f106c1552c0 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 0x7f106c0cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 0x7f106c0cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39912/0 (socket says 172.21.15.44:39912) 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106b7fe640 1 -- 172.21.15.44:0/3337544708 learned_addr learned my addr 172.21.15.44:0/3337544708 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106affd640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f106c0cc920 0x7f106c1e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106bfff640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c1552c0 0x7f106c1f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106b7fe640 1 -- 172.21.15.44:0/3337544708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c1552c0 msgr2=0x7f106c1f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106b7fe640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c1552c0 0x7f106c1f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106b7fe640 1 -- 172.21.15.44:0/3337544708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f106c0cc920 msgr2=0x7f106c1e7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106b7fe640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f106c0cc920 0x7f106c1e7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106b7fe640 1 -- 172.21.15.44:0/3337544708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106c1578c0 con 0x7f106c0caec0 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106affd640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f106c0cc920 0x7f106c1e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106bfff640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c1552c0 0x7f106c1f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:12.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.912+0000 7f106b7fe640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 0x7f106c0cd480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c3c0 tx=0x7f105c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.913+0000 7f1053fff640 1 -- 172.21.15.44:0/3337544708 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c0052d0 con 0x7f106c0caec0 2024-04-22T07:04:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.913+0000 7f1053fff640 1 -- 172.21.15.44:0/3337544708 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f105c005490 con 0x7f106c0caec0 2024-04-22T07:04:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.913+0000 7f1071b87640 1 -- 172.21.15.44:0/3337544708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106c156a60 con 0x7f106c0caec0 2024-04-22T07:04:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.913+0000 7f1053fff640 1 -- 172.21.15.44:0/3337544708 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f105c007260 con 0x7f106c0caec0 2024-04-22T07:04:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.913+0000 7f1071b87640 1 -- 172.21.15.44:0/3337544708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106c157020 con 0x7f106c0caec0 2024-04-22T07:04:12.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.914+0000 7f1071b87640 1 -- 172.21.15.44:0/3337544708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f106c068b00 con 0x7f106c0caec0 2024-04-22T07:04:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.914+0000 7f1053fff640 1 -- 172.21.15.44:0/3337544708 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f105c007400 con 0x7f106c0caec0 2024-04-22T07:04:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.915+0000 7f1053fff640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f105806c8f0 0x7f105806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.915+0000 7f1053fff640 1 -- 172.21.15.44:0/3337544708 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f105c08b050 con 0x7f106c0caec0 2024-04-22T07:04:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.915+0000 7f106affd640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f105806c8f0 0x7f105806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:12.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.915+0000 7f106affd640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f105806c8f0 0x7f105806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1060036000 tx=0x7f106003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:12.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:12.919+0000 7f1053fff640 1 -- 172.21.15.44:0/3337544708 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f105c059440 con 0x7f106c0caec0 2024-04-22T07:04:13.141 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.142+0000 7f1071b87640 1 -- 172.21.15.44:0/3337544708 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f106c0ca970 con 0x7f106c0caec0 2024-04-22T07:04:13.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.789+0000 7f1053fff640 1 -- 172.21.15.44:0/3337544708 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f105c058d40 con 0x7f106c0caec0 2024-04-22T07:04:13.789 INFO:teuthology.orchestra.run.smithi044.stderr:new fs with metadata pool 2 and data pool 3 2024-04-22T07:04:13.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.796+0000 7f1071b87640 1 -- 172.21.15.44:0/3337544708 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f105806c8f0 msgr2=0x7f105806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.796+0000 7f1071b87640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f105806c8f0 0x7f105806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1060036000 tx=0x7f106003b040 comp rx=0 tx=0).stop 2024-04-22T07:04:13.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.796+0000 7f1071b87640 1 -- 172.21.15.44:0/3337544708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 msgr2=0x7f106c0cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.796+0000 7f1071b87640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 0x7f106c0cd480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f105c00c3c0 tx=0x7f105c00c880 comp rx=0 tx=0).stop 2024-04-22T07:04:13.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.800+0000 7f1071b87640 1 -- 172.21.15.44:0/3337544708 shutdown_connections 2024-04-22T07:04:13.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.800+0000 7f1071b87640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f105806c8f0 0x7f105806edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.800+0000 7f1071b87640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f106c1552c0 0x7f106c1f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.800+0000 7f1071b87640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f106c0cc920 0x7f106c1e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.800+0000 7f1071b87640 1 --2- 172.21.15.44:0/3337544708 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f106c0caec0 0x7f106c0cd480 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.800+0000 7f1071b87640 1 -- 172.21.15.44:0/3337544708 >> 172.21.15.44:0/3337544708 conn(0x7f106c0676b0 msgr2=0x7f106c0ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:13.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.801+0000 7f1071b87640 1 -- 172.21.15.44:0/3337544708 shutdown_connections 2024-04-22T07:04:13.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.801+0000 7f1071b87640 1 -- 172.21.15.44:0/3337544708 wait complete. 2024-04-22T07:04:13.816 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:13.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.924+0000 7ff128ee0640 1 Processor -- start 2024-04-22T07:04:13.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.924+0000 7ff128ee0640 1 -- start start 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff128ee0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff124154f50 0x7ff12415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff128ee0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240ccf30 0x7ff1240cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff128ee0640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff12415bc20 con 0x7ff1240cc880 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff128ee0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff12415f1b0 con 0x7ff124154f50 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff128ee0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff1240cb7c0 con 0x7ff1240ccf30 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff121d9b640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff1240cc880 0x7ff1240c54c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37976/0 (socket says 172.21.15.44:37976) 2024-04-22T07:04:13.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff121d9b640 1 -- 172.21.15.44:0/2431128073 learned_addr learned my addr 172.21.15.44:0/2431128073 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff12259c640 1 --2- 172.21.15.44:0/2431128073 >> v2:172.21.15.70:3300/0 conn(0x7ff124154f50 0x7ff12415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff122d9d640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240ccf30 0x7ff1240cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160372859 0 0) 0x7ff10c001110 con 0x7ff1240cc880 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.925+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff110003520 con 0x7ff1240cc880 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 662457910 0 0) 0x7ff110003520 con 0x7ff1240cc880 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240ccf30 msgr2=0x7ff1240cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240ccf30 0x7ff1240cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 >> v2:172.21.15.70:3300/0 conn(0x7ff124154f50 msgr2=0x7ff12415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 --2- 172.21.15.44:0/2431128073 >> v2:172.21.15.70:3300/0 conn(0x7ff124154f50 0x7ff12415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12415e3f0 con 0x7ff1240cc880 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3718881788 0 0) 0x7ff10c001840 con 0x7ff1240cc880 2024-04-22T07:04:13.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 >> v1:172.21.15.44:6789/0 conn(0x7ff1240cc880 legacy=0x7ff1240c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1100039b0 0x7ff110003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff110004800 0x7ff110004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240ccf30 0x7ff1100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff12415f1b0 con 0x7ff110004800 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff1240cb7c0 con 0x7ff1100039b0 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff11000b9c0 con 0x7ff1240ccf30 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff121d9b640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff110004800 0x7ff110004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff12259c640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1100039b0 0x7ff110003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff122d9d640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240ccf30 0x7ff1100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff121d9b640 1 -- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240ccf30 msgr2=0x7ff1100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff121d9b640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240ccf30 0x7ff1100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff121d9b640 1 -- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1100039b0 msgr2=0x7ff110003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:13.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff121d9b640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1100039b0 0x7ff110003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.926+0000 7ff121d9b640 1 -- 172.21.15.44:0/2431128073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff110003520 con 0x7ff110004800 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.927+0000 7ff121d9b640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff110004800 0x7ff110004c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00c4c0 tx=0x7ff110005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853eee6db835485f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.927+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c00c0e0 con 0x7ff110004800 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.927+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff10c030070 con 0x7ff110004800 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.927+0000 7ff12159a640 1 -- 172.21.15.44:0/2431128073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0058d0 con 0x7ff110004800 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.928+0000 7ff128ee0640 1 -- 172.21.15.44:0/2431128073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff110004800 msgr2=0x7ff110004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.928+0000 7ff128ee0640 1 --2- 172.21.15.44:0/2431128073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff110004800 0x7ff110004c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00c4c0 tx=0x7ff110005830 comp rx=0 tx=0).stop 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.928+0000 7ff122d9d640 1 -- 172.21.15.44:0/2431128073 reap_dead start 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.928+0000 7ff128ee0640 1 -- 172.21.15.44:0/2431128073 shutdown_connections 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.928+0000 7ff128ee0640 1 -- 172.21.15.44:0/2431128073 >> 172.21.15.44:0/2431128073 conn(0x7ff124067d40 msgr2=0x7ff12406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.928+0000 7ff128ee0640 1 -- 172.21.15.44:0/2431128073 shutdown_connections 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.928+0000 7ff128ee0640 1 -- 172.21.15.44:0/2431128073 wait complete. 2024-04-22T07:04:13.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.929+0000 7ff128ee0640 1 Processor -- start 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.929+0000 7ff128ee0640 1 -- start start 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff128ee0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff110004800 0x7ff1241f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff128ee0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240cfc80 0x7ff1241e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff128ee0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1241f22d0 0x7ff1241f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff128ee0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff12415ec90 con 0x7ff1241f22d0 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff128ee0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff12415e9b0 con 0x7ff110004800 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff128ee0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff124155330 con 0x7ff1240cfc80 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff122d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1241f22d0 0x7ff1241f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff122d9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1241f22d0 0x7ff1241f2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39940/0 (socket says 172.21.15.44:39940) 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff122d9d640 1 -- 172.21.15.44:0/1304205488 learned_addr learned my addr 172.21.15.44:0/1304205488 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff12259c640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff110004800 0x7ff1241f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff121d9b640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240cfc80 0x7ff1241e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff122d9d640 1 -- 172.21.15.44:0/1304205488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240cfc80 msgr2=0x7ff1241e73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff122d9d640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240cfc80 0x7ff1241e73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff122d9d640 1 -- 172.21.15.44:0/1304205488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff110004800 msgr2=0x7ff1241f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff122d9d640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff110004800 0x7ff1241f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff122d9d640 1 -- 172.21.15.44:0/1304205488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1241f2c90 con 0x7ff1241f22d0 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff121d9b640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240cfc80 0x7ff1241e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.930+0000 7ff122d9d640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1241f22d0 0x7ff1241f2750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1240cce80 tx=0x7ff11400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.931+0000 7ff11affd640 1 -- 172.21.15.44:0/1304205488 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11400fcb0 con 0x7ff1241f22d0 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.931+0000 7ff11affd640 1 -- 172.21.15.44:0/1304205488 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11401d070 con 0x7ff1241f22d0 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.931+0000 7ff128ee0640 1 -- 172.21.15.44:0/1304205488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1240ce570 con 0x7ff1241f22d0 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.931+0000 7ff11affd640 1 -- 172.21.15.44:0/1304205488 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114015670 con 0x7ff1241f22d0 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.931+0000 7ff128ee0640 1 -- 172.21.15.44:0/1304205488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1240ceb30 con 0x7ff1241f22d0 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.932+0000 7ff128ee0640 1 -- 172.21.15.44:0/1304205488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1240ce300 con 0x7ff1241f22d0 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.932+0000 7ff11affd640 1 -- 172.21.15.44:0/1304205488 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff114015810 con 0x7ff1241f22d0 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.932+0000 7ff11affd640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff10406c920 0x7ff10406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:13.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.933+0000 7ff11affd640 1 -- 172.21.15.44:0/1304205488 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff11408c2d0 con 0x7ff1241f22d0 2024-04-22T07:04:13.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.933+0000 7ff12259c640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff10406c920 0x7ff10406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:13.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.933+0000 7ff12259c640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff10406c920 0x7ff10406ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009bc0 tx=0x7ff11c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:13.942 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:13.937+0000 7ff11affd640 1 -- 172.21.15.44:0/1304205488 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1241f2c90 con 0x7ff1241f22d0 2024-04-22T07:04:14.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.122+0000 7ff128ee0640 1 -- 172.21.15.44:0/1304205488 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff12404fdc0 con 0x7ff1241f22d0 2024-04-22T07:04:14.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.123+0000 7ff11affd640 1 -- 172.21.15.44:0/1304205488 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16570 (secure 0 0 0) 0x7ff114010040 con 0x7ff1241f22d0 2024-04-22T07:04:14.122 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:14.122 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":22,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:13.144539+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 -- 172.21.15.44:0/1304205488 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff10406c920 msgr2=0x7ff10406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff10406c920 0x7ff10406ede0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009bc0 tx=0x7ff11c009190 comp rx=0 tx=0).stop 2024-04-22T07:04:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 -- 172.21.15.44:0/1304205488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1241f22d0 msgr2=0x7ff1241f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1241f22d0 0x7ff1241f2750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1240cce80 tx=0x7ff11400f1b0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 -- 172.21.15.44:0/1304205488 shutdown_connections 2024-04-22T07:04:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff10406c920 0x7ff10406ede0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.124 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1240cfc80 0x7ff1241e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff1241f22d0 0x7ff1241f2750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 --2- 172.21.15.44:0/1304205488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff110004800 0x7ff1241f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 -- 172.21.15.44:0/1304205488 >> 172.21.15.44:0/1304205488 conn(0x7ff124067d40 msgr2=0x7ff124158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 -- 172.21.15.44:0/1304205488 shutdown_connections 2024-04-22T07:04:14.125 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.125+0000 7ff128ee0640 1 -- 172.21.15.44:0/1304205488 wait complete. 2024-04-22T07:04:14.135 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:14.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.239+0000 7f382339d640 1 Processor -- start 2024-04-22T07:04:14.238 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.240+0000 7f382339d640 1 -- start start 2024-04-22T07:04:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.240+0000 7f382339d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f381c1551a0 0x7f381c15ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.240+0000 7f382339d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c5380 0x7f381c070080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.240+0000 7f382339d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f381c06ba80 con 0x7f381c0c33f0 2024-04-22T07:04:14.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.240+0000 7f382339d640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f381c15f350 con 0x7f381c1551a0 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.240+0000 7f382339d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f381c06c2e0 con 0x7f381c0c5380 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.240+0000 7f3821139640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f381c1551a0 0x7f381c15ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.240+0000 7f3821139640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f381c1551a0 0x7f381c15ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39948/0 (socket says 172.21.15.44:39948) 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.240+0000 7f3821139640 1 -- 172.21.15.44:0/226836978 learned_addr learned my addr 172.21.15.44:0/226836978 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.241+0000 7f382193a640 1 --2- 172.21.15.44:0/226836978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c5380 0x7f381c070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.241+0000 7f3821139640 1 -- 172.21.15.44:0/226836978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c5380 msgr2=0x7f381c070080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.241+0000 7f3821139640 1 --2- 172.21.15.44:0/226836978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c5380 0x7f381c070080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.241+0000 7f3821139640 1 -- 172.21.15.44:0/226836978 >> v1:172.21.15.70:6789/0 conn(0x7f381c0c33f0 legacy=0x7f381c15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.241+0000 7f3821139640 1 -- 172.21.15.44:0/226836978 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c06bd20 con 0x7f381c1551a0 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.241+0000 7f3821139640 1 --2- 172.21.15.44:0/226836978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c1551a0 0x7f381c15ee10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f381c0cbb80 tx=0x7f381402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b817eaa5741934e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f381bfff640 1 -- 172.21.15.44:0/226836978 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814034070 con 0x7f381c1551a0 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f381bfff640 1 -- 172.21.15.44:0/226836978 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3814030840 con 0x7f381c1551a0 2024-04-22T07:04:14.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f381bfff640 1 -- 172.21.15.44:0/226836978 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814030c10 con 0x7f381c1551a0 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f382339d640 1 -- 172.21.15.44:0/226836978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c1551a0 msgr2=0x7f381c15ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f382339d640 1 --2- 172.21.15.44:0/226836978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c1551a0 0x7f381c15ee10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f381c0cbb80 tx=0x7f381402b980 comp rx=0 tx=0).stop 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f382339d640 1 -- 172.21.15.44:0/226836978 shutdown_connections 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f382339d640 1 --2- 172.21.15.44:0/226836978 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c5380 0x7f381c070080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f382339d640 1 --2- 172.21.15.44:0/226836978 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c1551a0 0x7f381c15ee10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f382339d640 1 -- 172.21.15.44:0/226836978 >> 172.21.15.44:0/226836978 conn(0x7f381c067ce0 msgr2=0x7f381c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f382339d640 1 -- 172.21.15.44:0/226836978 shutdown_connections 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.242+0000 7f382339d640 1 -- 172.21.15.44:0/226836978 wait complete. 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.243+0000 7f382339d640 1 Processor -- start 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.243+0000 7f382339d640 1 -- start start 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f382339d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c33f0 0x7f381c153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f382339d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c0c5380 0x7f381c152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f382339d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c1551a0 0x7f381c0c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f382339d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f381c15b380 con 0x7f381c0c5380 2024-04-22T07:04:14.245 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f382339d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f381c15b0a0 con 0x7f381c1551a0 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f382339d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f381c155580 con 0x7f381c0c33f0 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f3820938640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c0c5380 0x7f381c152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f3820938640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c0c5380 0x7f381c152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39958/0 (socket says 172.21.15.44:39958) 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f3820938640 1 -- 172.21.15.44:0/2821376325 learned_addr learned my addr 172.21.15.44:0/2821376325 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f3821139640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c33f0 0x7f381c153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.244+0000 7f382193a640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c1551a0 0x7f381c0c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f3820938640 1 -- 172.21.15.44:0/2821376325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c33f0 msgr2=0x7f381c153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f3820938640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c33f0 0x7f381c153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f3820938640 1 -- 172.21.15.44:0/2821376325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c1551a0 msgr2=0x7f381c0c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f3820938640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c1551a0 0x7f381c0c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f3820938640 1 -- 172.21.15.44:0/2821376325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c1f52e0 con 0x7f381c0c5380 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f3821139640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c33f0 0x7f381c153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:14.246 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f3820938640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c0c5380 0x7f381c152360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f381000c400 tx=0x7f381000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f3819ffb640 1 -- 172.21.15.44:0/2821376325 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810011790 con 0x7f381c0c5380 2024-04-22T07:04:14.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f3819ffb640 1 -- 172.21.15.44:0/2821376325 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3810017070 con 0x7f381c0c5380 2024-04-22T07:04:14.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f3819ffb640 1 -- 172.21.15.44:0/2821376325 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810011bb0 con 0x7f381c0c5380 2024-04-22T07:04:14.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f382339d640 1 -- 172.21.15.44:0/2821376325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381c1515c0 con 0x7f381c0c5380 2024-04-22T07:04:14.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.245+0000 7f382339d640 1 -- 172.21.15.44:0/2821376325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381c151880 con 0x7f381c0c5380 2024-04-22T07:04:14.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.246+0000 7f382339d640 1 -- 172.21.15.44:0/2821376325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f381c069210 con 0x7f381c0c5380 2024-04-22T07:04:14.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.247+0000 7f3819ffb640 1 -- 172.21.15.44:0/2821376325 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f381001c460 con 0x7f381c0c5380 2024-04-22T07:04:14.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.247+0000 7f3819ffb640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f380806c8f0 0x7f380806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.247+0000 7f3819ffb640 1 -- 172.21.15.44:0/2821376325 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f381008ba50 con 0x7f381c0c5380 2024-04-22T07:04:14.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.247+0000 7f3821139640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f380806c8f0 0x7f380806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.247+0000 7f3821139640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f380806c8f0 0x7f380806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3814036000 tx=0x7f381403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.250 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.251+0000 7f3819ffb640 1 -- 172.21.15.44:0/2821376325 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f381c069210 con 0x7f381c0c5380 2024-04-22T07:04:14.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.440+0000 7f382339d640 1 -- 172.21.15.44:0/2821376325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f381c069630 con 0x7f381c0c5380 2024-04-22T07:04:14.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.441+0000 7f3819ffb640 1 -- 172.21.15.44:0/2821376325 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16570 (secure 0 0 0) 0x7f381c069630 con 0x7f381c0c5380 2024-04-22T07:04:14.440 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:14.440 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":22,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:13.144539+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 -- 172.21.15.44:0/2821376325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f380806c8f0 msgr2=0x7f380806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f380806c8f0 0x7f380806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3814036000 tx=0x7f381403b040 comp rx=0 tx=0).stop 2024-04-22T07:04:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 -- 172.21.15.44:0/2821376325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c0c5380 msgr2=0x7f381c152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c0c5380 0x7f381c152360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f381000c400 tx=0x7f381000c8c0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 -- 172.21.15.44:0/2821376325 shutdown_connections 2024-04-22T07:04:14.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f380806c8f0 0x7f380806edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f381c1551a0 0x7f381c0c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f381c0c5380 0x7f381c152360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 --2- 172.21.15.44:0/2821376325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f381c0c33f0 0x7f381c153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 -- 172.21.15.44:0/2821376325 >> 172.21.15.44:0/2821376325 conn(0x7f381c067ce0 msgr2=0x7f381c0c4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 -- 172.21.15.44:0/2821376325 shutdown_connections 2024-04-22T07:04:14.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.443+0000 7f382339d640 1 -- 172.21.15.44:0/2821376325 wait complete. 2024-04-22T07:04:14.453 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-22T07:04:14.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.557+0000 7fef67006640 1 Processor -- start 2024-04-22T07:04:14.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.557+0000 7fef67006640 1 -- start start 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef67006640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fef6006d5d0 0x7fef600c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef67006640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fef60071770 0x7fef60157940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef67006640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef600c5560 con 0x7fef6006d9b0 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef67006640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fef600c5840 con 0x7fef60071770 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef67006640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fef600c5b20 con 0x7fef6006d5d0 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef655a3640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fef60071770 0x7fef60157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef655a3640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fef60071770 0x7fef60157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39966/0 (socket says 172.21.15.44:39966) 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef57fff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fef6006d9b0 0x7fef60151530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37978/0 (socket says 172.21.15.44:37978) 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef655a3640 1 -- 172.21.15.44:0/242227805 learned_addr learned my addr 172.21.15.44:0/242227805 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef64da2640 1 --2- 172.21.15.44:0/242227805 >> v2:172.21.15.70:3300/0 conn(0x7fef6006d5d0 0x7fef600c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef655a3640 1 -- 172.21.15.44:0/242227805 >> v2:172.21.15.70:3300/0 conn(0x7fef6006d5d0 msgr2=0x7fef600c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef655a3640 1 --2- 172.21.15.44:0/242227805 >> v2:172.21.15.70:3300/0 conn(0x7fef6006d5d0 0x7fef600c3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.558+0000 7fef655a3640 1 -- 172.21.15.44:0/242227805 >> v1:172.21.15.44:6789/0 conn(0x7fef6006d9b0 legacy=0x7fef60151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.559+0000 7fef655a3640 1 -- 172.21.15.44:0/242227805 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef60073130 con 0x7fef60071770 2024-04-22T07:04:14.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.559+0000 7fef577fe640 1 -- 172.21.15.44:0/242227805 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 789898908 0 0) 0x7fef600c5560 con 0x7fef6006d9b0 2024-04-22T07:04:14.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.559+0000 7fef655a3640 1 --2- 172.21.15.44:0/242227805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef60071770 0x7fef60157940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fef58009e50 tx=0x7fef5802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c52aab51d41aead server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.559+0000 7fef577fe640 1 -- 172.21.15.44:0/242227805 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5802cbb0 con 0x7fef60071770 2024-04-22T07:04:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.559+0000 7fef577fe640 1 -- 172.21.15.44:0/242227805 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5802cd50 con 0x7fef60071770 2024-04-22T07:04:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.559+0000 7fef577fe640 1 -- 172.21.15.44:0/242227805 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58034610 con 0x7fef60071770 2024-04-22T07:04:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.560+0000 7fef67006640 1 -- 172.21.15.44:0/242227805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef60071770 msgr2=0x7fef60157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.560+0000 7fef67006640 1 --2- 172.21.15.44:0/242227805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef60071770 0x7fef60157940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fef58009e50 tx=0x7fef5802c010 comp rx=0 tx=0).stop 2024-04-22T07:04:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.560+0000 7fef67006640 1 -- 172.21.15.44:0/242227805 shutdown_connections 2024-04-22T07:04:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.560+0000 7fef67006640 1 --2- 172.21.15.44:0/242227805 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef60071770 0x7fef60157940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.560+0000 7fef67006640 1 --2- 172.21.15.44:0/242227805 >> v2:172.21.15.70:3300/0 conn(0x7fef6006d5d0 0x7fef600c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.560+0000 7fef67006640 1 -- 172.21.15.44:0/242227805 >> 172.21.15.44:0/242227805 conn(0x7fef600c9020 msgr2=0x7fef600c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:14.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.560+0000 7fef67006640 1 -- 172.21.15.44:0/242227805 shutdown_connections 2024-04-22T07:04:14.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.560+0000 7fef67006640 1 -- 172.21.15.44:0/242227805 wait complete. 2024-04-22T07:04:14.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.561+0000 7fef67006640 1 Processor -- start 2024-04-22T07:04:14.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.561+0000 7fef67006640 1 -- start start 2024-04-22T07:04:14.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef67006640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef6006d5d0 0x7fef600cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef67006640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef6006d9b0 0x7fef600cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef67006640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60071770 0x7fef601467d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef67006640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fef600c3b40 con 0x7fef6006d5d0 2024-04-22T07:04:14.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef67006640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fef600736f0 con 0x7fef60071770 2024-04-22T07:04:14.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef67006640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fef600c5200 con 0x7fef6006d9b0 2024-04-22T07:04:14.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef64da2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef6006d5d0 0x7fef600cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef64da2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef6006d5d0 0x7fef600cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:39982/0 (socket says 172.21.15.44:39982) 2024-04-22T07:04:14.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef64da2640 1 -- 172.21.15.44:0/3952303407 learned_addr learned my addr 172.21.15.44:0/3952303407 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:14.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef655a3640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60071770 0x7fef601467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef57fff640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef6006d9b0 0x7fef600cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef64da2640 1 -- 172.21.15.44:0/3952303407 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef6006d9b0 msgr2=0x7fef600cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef64da2640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef6006d9b0 0x7fef600cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef64da2640 1 -- 172.21.15.44:0/3952303407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60071770 msgr2=0x7fef601467d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef64da2640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60071770 0x7fef601467d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef64da2640 1 -- 172.21.15.44:0/3952303407 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef60142220 con 0x7fef6006d5d0 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.562+0000 7fef655a3640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60071770 0x7fef601467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.563+0000 7fef64da2640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef6006d5d0 0x7fef600cb1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00e9e0 tx=0x7fef5c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.563+0000 7fef557fa640 1 -- 172.21.15.44:0/3952303407 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c017070 con 0x7fef6006d5d0 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.563+0000 7fef557fa640 1 -- 172.21.15.44:0/3952303407 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef5c00c9d0 con 0x7fef6006d5d0 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.563+0000 7fef67006640 1 -- 172.21.15.44:0/3952303407 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef601ef030 con 0x7fef6006d5d0 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.563+0000 7fef557fa640 1 -- 172.21.15.44:0/3952303407 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00cdc0 con 0x7fef6006d5d0 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.563+0000 7fef67006640 1 -- 172.21.15.44:0/3952303407 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef60145cf0 con 0x7fef6006d5d0 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.564+0000 7fef67006640 1 -- 172.21.15.44:0/3952303407 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef60068b00 con 0x7fef6006d5d0 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.564+0000 7fef557fa640 1 -- 172.21.15.44:0/3952303407 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef5c01a070 con 0x7fef6006d5d0 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.565+0000 7fef557fa640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fef4c06c8f0 0x7fef4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:14.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.565+0000 7fef557fa640 1 -- 172.21.15.44:0/3952303407 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fef5c08bb30 con 0x7fef6006d5d0 2024-04-22T07:04:14.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.565+0000 7fef57fff640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fef4c06c8f0 0x7fef4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:14.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.565+0000 7fef57fff640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fef4c06c8f0 0x7fef4c06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef600c6d90 tx=0x7fef50007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:14.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.569+0000 7fef557fa640 1 -- 172.21.15.44:0/3952303407 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef5c059f60 con 0x7fef6006d5d0 2024-04-22T07:04:14.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:14.793+0000 7fef67006640 1 -- 172.21.15.44:0/3952303407 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fef60150fc0 con 0x7fef6006d5d0 2024-04-22T07:04:15.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.795+0000 7fef557fa640 1 -- 172.21.15.44:0/3952303407 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7fef5c059860 con 0x7fef6006d5d0 2024-04-22T07:04:15.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 -- 172.21.15.44:0/3952303407 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fef4c06c8f0 msgr2=0x7fef4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fef4c06c8f0 0x7fef4c06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef600c6d90 tx=0x7fef50007b50 comp rx=0 tx=0).stop 2024-04-22T07:04:15.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 -- 172.21.15.44:0/3952303407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef6006d5d0 msgr2=0x7fef600cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef6006d5d0 0x7fef600cb1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00e9e0 tx=0x7fef5c00eea0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 -- 172.21.15.44:0/3952303407 shutdown_connections 2024-04-22T07:04:15.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fef4c06c8f0 0x7fef4c06edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fef6006d9b0 0x7fef600cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef60071770 0x7fef601467d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 --2- 172.21.15.44:0/3952303407 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fef6006d5d0 0x7fef600cb1e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 -- 172.21.15.44:0/3952303407 >> 172.21.15.44:0/3952303407 conn(0x7fef600c9020 msgr2=0x7fef60141780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:15.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 -- 172.21.15.44:0/3952303407 shutdown_connections 2024-04-22T07:04:15.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.798+0000 7fef67006640 1 -- 172.21.15.44:0/3952303407 wait complete. 2024-04-22T07:04:15.810 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300} 2024-04-22T07:04:15.810 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-22T07:04:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.921+0000 7f5644aba640 1 Processor -- start 2024-04-22T07:04:15.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.921+0000 7f5644aba640 1 -- start start 2024-04-22T07:04:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.922+0000 7f5644aba640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5640154fb0 0x7f56400c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.922+0000 7f5644aba640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400c5870 0x7f564015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.922+0000 7f5644aba640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f56400cca20 0x7f564006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.922+0000 7f5644aba640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f564015e960 con 0x7f5640154fb0 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.922+0000 7f5644aba640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f56400cb610 con 0x7f56400cca20 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.922+0000 7f5644aba640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f56400cb8f0 con 0x7f56400c5870 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.922+0000 7f563e59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5640154fb0 0x7f56400c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.922+0000 7f563e59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5640154fb0 0x7f56400c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37728/0 (socket says 172.21.15.44:37728) 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.922+0000 7f563e59c640 1 -- 172.21.15.44:0/3895471225 learned_addr learned my addr 172.21.15.44:0/3895471225 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.923+0000 7f563ed9d640 1 --2- 172.21.15.44:0/3895471225 >> v2:172.21.15.70:3300/0 conn(0x7f56400cca20 0x7f564006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.923+0000 7f563dd9b640 1 --2- 172.21.15.44:0/3895471225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400c5870 0x7f564015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.923+0000 7f563e59c640 1 -- 172.21.15.44:0/3895471225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400c5870 msgr2=0x7f564015edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.923+0000 7f563e59c640 1 --2- 172.21.15.44:0/3895471225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400c5870 0x7f564015edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.923+0000 7f563e59c640 1 -- 172.21.15.44:0/3895471225 >> v2:172.21.15.70:3300/0 conn(0x7f56400cca20 msgr2=0x7f564006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.923+0000 7f563e59c640 1 --2- 172.21.15.44:0/3895471225 >> v2:172.21.15.70:3300/0 conn(0x7f56400cca20 0x7f564006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.923+0000 7f563e59c640 1 -- 172.21.15.44:0/3895471225 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564015e310 con 0x7f5640154fb0 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.923+0000 7f563dd9b640 1 --2- 172.21.15.44:0/3895471225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400c5870 0x7f564015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:15.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.923+0000 7f563e59c640 1 --2- 172.21.15.44:0/3895471225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 0x7f56400c5dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f56400c8660 tx=0x7f563802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d961ea6371421abb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f563d59a640 1 -- 172.21.15.44:0/3895471225 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638034070 con 0x7f5640154fb0 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f563d59a640 1 -- 172.21.15.44:0/3895471225 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5638030850 con 0x7f5640154fb0 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f563d59a640 1 -- 172.21.15.44:0/3895471225 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638030c20 con 0x7f5640154fb0 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f5644aba640 1 -- 172.21.15.44:0/3895471225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 msgr2=0x7f56400c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f5644aba640 1 --2- 172.21.15.44:0/3895471225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 0x7f56400c5dc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f56400c8660 tx=0x7f563802b990 comp rx=0 tx=0).stop 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f5644aba640 1 -- 172.21.15.44:0/3895471225 shutdown_connections 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f5644aba640 1 --2- 172.21.15.44:0/3895471225 >> v2:172.21.15.70:3300/0 conn(0x7f56400cca20 0x7f564006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f5644aba640 1 --2- 172.21.15.44:0/3895471225 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400c5870 0x7f564015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f5644aba640 1 --2- 172.21.15.44:0/3895471225 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 0x7f56400c5dc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f5644aba640 1 -- 172.21.15.44:0/3895471225 >> 172.21.15.44:0/3895471225 conn(0x7f5640068100 msgr2=0x7f56400d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f5644aba640 1 -- 172.21.15.44:0/3895471225 shutdown_connections 2024-04-22T07:04:15.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.924+0000 7f5644aba640 1 -- 172.21.15.44:0/3895471225 wait complete. 2024-04-22T07:04:15.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.926+0000 7f5644aba640 1 Processor -- start 2024-04-22T07:04:15.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.926+0000 7f5644aba640 1 -- start start 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.926+0000 7f5644aba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56400c5870 0x7f564015a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.926+0000 7f5644aba640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400cca20 0x7f564015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.926+0000 7f5644aba640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 0x7f56401f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.926+0000 7f5644aba640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5640155510 con 0x7f5640154fb0 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.926+0000 7f5644aba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5640155390 con 0x7f56400c5870 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.926+0000 7f5644aba640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f564015b820 con 0x7f56400cca20 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 0x7f56401f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 0x7f56401f0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37742/0 (socket says 172.21.15.44:37742) 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563ed9d640 1 -- 172.21.15.44:0/1098550743 learned_addr learned my addr 172.21.15.44:0/1098550743 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563dd9b640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400cca20 0x7f564015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563e59c640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56400c5870 0x7f564015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563ed9d640 1 -- 172.21.15.44:0/1098550743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400cca20 msgr2=0x7f564015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563ed9d640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400cca20 0x7f564015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563ed9d640 1 -- 172.21.15.44:0/1098550743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56400c5870 msgr2=0x7f564015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563ed9d640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56400c5870 0x7f564015a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563ed9d640 1 -- 172.21.15.44:0/1098550743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564014bd40 con 0x7f5640154fb0 2024-04-22T07:04:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f563ed9d640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 0x7f56401f0c10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5630007e40 tx=0x7f563000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f5636ffd640 1 -- 172.21.15.44:0/1098550743 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630014620 con 0x7f5640154fb0 2024-04-22T07:04:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f5636ffd640 1 -- 172.21.15.44:0/1098550743 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5630019070 con 0x7f5640154fb0 2024-04-22T07:04:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f5644aba640 1 -- 172.21.15.44:0/1098550743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56400ce620 con 0x7f5640154fb0 2024-04-22T07:04:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f5636ffd640 1 -- 172.21.15.44:0/1098550743 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630014a20 con 0x7f5640154fb0 2024-04-22T07:04:15.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.927+0000 7f5644aba640 1 -- 172.21.15.44:0/1098550743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56400cebe0 con 0x7f5640154fb0 2024-04-22T07:04:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.928+0000 7f5644aba640 1 -- 172.21.15.44:0/1098550743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56400cf2a0 con 0x7f5640154fb0 2024-04-22T07:04:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.929+0000 7f5636ffd640 1 -- 172.21.15.44:0/1098550743 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5630014bc0 con 0x7f5640154fb0 2024-04-22T07:04:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.929+0000 7f5636ffd640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f562006c8f0 0x7f562006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:15.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.929+0000 7f5636ffd640 1 -- 172.21.15.44:0/1098550743 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f563008c2c0 con 0x7f5640154fb0 2024-04-22T07:04:15.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.929+0000 7f563e59c640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f562006c8f0 0x7f562006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:15.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.930+0000 7f563e59c640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f562006c8f0 0x7f562006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5638036000 tx=0x7f563803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:15.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:15.933+0000 7f5636ffd640 1 -- 172.21.15.44:0/1098550743 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f56400cf2a0 con 0x7f5640154fb0 2024-04-22T07:04:16.158 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.159+0000 7f5644aba640 1 -- 172.21.15.44:0/1098550743 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f56401546c0 con 0x7f5640154fb0 2024-04-22T07:04:16.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.797+0000 7f5636ffd640 1 -- 172.21.15.44:0/1098550743 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7f56401546c0 con 0x7f5640154fb0 2024-04-22T07:04:16.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.799+0000 7f5644aba640 1 -- 172.21.15.44:0/1098550743 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f562006c8f0 msgr2=0x7f562006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:16.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.799+0000 7f5644aba640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f562006c8f0 0x7f562006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5638036000 tx=0x7f563803b040 comp rx=0 tx=0).stop 2024-04-22T07:04:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.799+0000 7f5644aba640 1 -- 172.21.15.44:0/1098550743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 msgr2=0x7f56401f0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.799+0000 7f5644aba640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 0x7f56401f0c10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5630007e40 tx=0x7f563000e4b0 comp rx=0 tx=0).stop 2024-04-22T07:04:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.799+0000 7f5644aba640 1 -- 172.21.15.44:0/1098550743 shutdown_connections 2024-04-22T07:04:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.799+0000 7f5644aba640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f562006c8f0 0x7f562006edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.799+0000 7f5644aba640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f56400cca20 0x7f564015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.799+0000 7f5644aba640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5640154fb0 0x7f56401f0c10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.799+0000 7f5644aba640 1 --2- 172.21.15.44:0/1098550743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56400c5870 0x7f564015a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.799+0000 7f5644aba640 1 -- 172.21.15.44:0/1098550743 >> 172.21.15.44:0/1098550743 conn(0x7f5640068100 msgr2=0x7f5640153e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.800+0000 7f5644aba640 1 -- 172.21.15.44:0/1098550743 shutdown_connections 2024-04-22T07:04:16.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.800+0000 7f5644aba640 1 -- 172.21.15.44:0/1098550743 wait complete. 2024-04-22T07:04:16.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-22T07:04:16.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.922+0000 7fc99904e640 1 Processor -- start 2024-04-22T07:04:16.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.922+0000 7fc99904e640 1 -- start start 2024-04-22T07:04:16.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.923+0000 7fc99904e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc994155040 0x7fc994072c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:16.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.923+0000 7fc99904e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cd3d0 0x7fc9940cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:16.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.923+0000 7fc99904e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9940cdd60 con 0x7fc9940cbcc0 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.923+0000 7fc99904e640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fc9940cb270 con 0x7fc994155040 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.923+0000 7fc99904e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc99415af90 con 0x7fc9940cd3d0 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.923+0000 7fc992d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc994155040 0x7fc994072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.923+0000 7fc992d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc994155040 0x7fc994072c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37746/0 (socket says 172.21.15.44:37746) 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.923+0000 7fc99259c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fc9940cbcc0 0x7fc9940d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37964/0 (socket says 172.21.15.44:37964) 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.923+0000 7fc992d9d640 1 -- 172.21.15.44:0/3167361462 learned_addr learned my addr 172.21.15.44:0/3167361462 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.924+0000 7fc99359e640 1 --2- 172.21.15.44:0/3167361462 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cd3d0 0x7fc9940cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.924+0000 7fc992d9d640 1 -- 172.21.15.44:0/3167361462 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cd3d0 msgr2=0x7fc9940cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.924+0000 7fc992d9d640 1 --2- 172.21.15.44:0/3167361462 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cd3d0 0x7fc9940cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.924+0000 7fc992d9d640 1 -- 172.21.15.44:0/3167361462 >> v1:172.21.15.44:6789/0 conn(0x7fc9940cbcc0 legacy=0x7fc9940d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.924+0000 7fc991d9b640 1 -- 172.21.15.44:0/3167361462 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4116203912 0 0) 0x7fc9940cdd60 con 0x7fc9940cbcc0 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.924+0000 7fc992d9d640 1 -- 172.21.15.44:0/3167361462 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99415b270 con 0x7fc994155040 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.924+0000 7fc99359e640 1 --2- 172.21.15.44:0/3167361462 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cd3d0 0x7fc9940cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:16.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.924+0000 7fc992d9d640 1 --2- 172.21.15.44:0/3167361462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 0x7fc994072c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009870 tx=0x7fc98c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb8bbb5bf76e259 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:16.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc991d9b640 1 -- 172.21.15.44:0/3167361462 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c034070 con 0x7fc994155040 2024-04-22T07:04:16.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc991d9b640 1 -- 172.21.15.44:0/3167361462 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98c030840 con 0x7fc994155040 2024-04-22T07:04:16.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc991d9b640 1 -- 172.21.15.44:0/3167361462 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c030c10 con 0x7fc994155040 2024-04-22T07:04:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc99904e640 1 -- 172.21.15.44:0/3167361462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 msgr2=0x7fc994072c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc99904e640 1 --2- 172.21.15.44:0/3167361462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 0x7fc994072c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009870 tx=0x7fc98c02b980 comp rx=0 tx=0).stop 2024-04-22T07:04:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc99904e640 1 -- 172.21.15.44:0/3167361462 shutdown_connections 2024-04-22T07:04:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc99904e640 1 --2- 172.21.15.44:0/3167361462 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cd3d0 0x7fc9940cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc99904e640 1 --2- 172.21.15.44:0/3167361462 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 0x7fc994072c10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc99904e640 1 -- 172.21.15.44:0/3167361462 >> 172.21.15.44:0/3167361462 conn(0x7fc99406caa0 msgr2=0x7fc99406ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc99904e640 1 -- 172.21.15.44:0/3167361462 shutdown_connections 2024-04-22T07:04:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.925+0000 7fc99904e640 1 -- 172.21.15.44:0/3167361462 wait complete. 2024-04-22T07:04:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.927+0000 7fc99904e640 1 Processor -- start 2024-04-22T07:04:16.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.927+0000 7fc99904e640 1 -- start start 2024-04-22T07:04:16.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.927+0000 7fc99904e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cbcc0 0x7fc99415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.927+0000 7fc99904e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940cd3d0 0x7fc994156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.927+0000 7fc99904e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 0x7fc994157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.927+0000 7fc99904e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc99415bb10 con 0x7fc994155040 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.927+0000 7fc99904e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc99415b830 con 0x7fc9940cd3d0 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.927+0000 7fc99904e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc9940cceb0 con 0x7fc9940cbcc0 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 0x7fc994157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 0x7fc994157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37762/0 (socket says 172.21.15.44:37762) 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99359e640 1 -- 172.21.15.44:0/1580173295 learned_addr learned my addr 172.21.15.44:0/1580173295 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc992d9d640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cbcc0 0x7fc99415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99259c640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940cd3d0 0x7fc994156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99359e640 1 -- 172.21.15.44:0/1580173295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cbcc0 msgr2=0x7fc99415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99359e640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cbcc0 0x7fc99415eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99359e640 1 -- 172.21.15.44:0/1580173295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940cd3d0 msgr2=0x7fc994156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:16.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99359e640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940cd3d0 0x7fc994156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:16.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99359e640 1 -- 172.21.15.44:0/1580173295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99415f040 con 0x7fc994155040 2024-04-22T07:04:16.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99259c640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940cd3d0 0x7fc994156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:16.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99359e640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 0x7fc994157440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc984007f20 tx=0x7fc98400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:16.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc98b7fe640 1 -- 172.21.15.44:0/1580173295 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98401a070 con 0x7fc994155040 2024-04-22T07:04:16.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc98b7fe640 1 -- 172.21.15.44:0/1580173295 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc984003b90 con 0x7fc994155040 2024-04-22T07:04:16.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99904e640 1 -- 172.21.15.44:0/1580173295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99415a910 con 0x7fc994155040 2024-04-22T07:04:16.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc98b7fe640 1 -- 172.21.15.44:0/1580173295 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984004030 con 0x7fc994155040 2024-04-22T07:04:16.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.928+0000 7fc99904e640 1 -- 172.21.15.44:0/1580173295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941f0b80 con 0x7fc994155040 2024-04-22T07:04:16.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.929+0000 7fc99904e640 1 -- 172.21.15.44:0/1580173295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc994068b00 con 0x7fc994155040 2024-04-22T07:04:16.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.930+0000 7fc98b7fe640 1 -- 172.21.15.44:0/1580173295 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc98401d070 con 0x7fc994155040 2024-04-22T07:04:16.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.930+0000 7fc98b7fe640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc97406c920 0x7fc97406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:16.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.930+0000 7fc992d9d640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc97406c920 0x7fc97406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:16.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.931+0000 7fc98b7fe640 1 -- 172.21.15.44:0/1580173295 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc98408bf50 con 0x7fc994155040 2024-04-22T07:04:16.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.931+0000 7fc992d9d640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc97406c920 0x7fc97406ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc98c036000 tx=0x7fc98c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:16.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:16.935+0000 7fc98b7fe640 1 -- 172.21.15.44:0/1580173295 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc984010070 con 0x7fc994155040 2024-04-22T07:04:17.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.161+0000 7fc99904e640 1 -- 172.21.15.44:0/1580173295 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc9940c5ee0 con 0x7fc994155040 2024-04-22T07:04:17.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.802+0000 7fc98b7fe640 1 -- 172.21.15.44:0/1580173295 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7fc98405a250 con 0x7fc994155040 2024-04-22T07:04:17.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 -- 172.21.15.44:0/1580173295 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc97406c920 msgr2=0x7fc97406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc97406c920 0x7fc97406ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc98c036000 tx=0x7fc98c008030 comp rx=0 tx=0).stop 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 -- 172.21.15.44:0/1580173295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 msgr2=0x7fc994157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 0x7fc994157440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc984007f20 tx=0x7fc98400d560 comp rx=0 tx=0).stop 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 -- 172.21.15.44:0/1580173295 shutdown_connections 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc97406c920 0x7fc97406ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc994155040 0x7fc994157440 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9940cd3d0 0x7fc994156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 --2- 172.21.15.44:0/1580173295 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc9940cbcc0 0x7fc99415eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 -- 172.21.15.44:0/1580173295 >> 172.21.15.44:0/1580173295 conn(0x7fc99406caa0 msgr2=0x7fc9940cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 -- 172.21.15.44:0/1580173295 shutdown_connections 2024-04-22T07:04:17.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.806+0000 7fc99904e640 1 -- 172.21.15.44:0/1580173295 wait complete. 2024-04-22T07:04:17.818 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-22T07:04:17.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.929+0000 7f1a0c3f4640 1 Processor -- start 2024-04-22T07:04:17.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.929+0000 7f1a0c3f4640 1 -- start start 2024-04-22T07:04:17.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.930+0000 7f1a0c3f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a041552c0 0x7f1a040b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:17.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.930+0000 7f1a0c3f4640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a041556a0 con 0x7f1a040cc920 2024-04-22T07:04:17.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.930+0000 7f1a0c3f4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0415b160 con 0x7f1a040caec0 2024-04-22T07:04:17.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.930+0000 7f1a0c3f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0415b440 con 0x7f1a041552c0 2024-04-22T07:04:17.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.930+0000 7f1a0a991640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1a040cc920 0x7f1a040cd310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37980/0 (socket says 172.21.15.44:37980) 2024-04-22T07:04:17.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.930+0000 7f1a0a991640 1 -- 172.21.15.44:0/2244883210 learned_addr learned my addr 172.21.15.44:0/2244883210 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:17.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.930+0000 7f1a0a190640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a041552c0 0x7f1a040b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:17.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.930+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4202834774 0 0) 0x7f1a041556a0 con 0x7f1a040cc920 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.930+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19ec003520 con 0x7f1a040cc920 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.930+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2374249870 0 0) 0x7f19ec003520 con 0x7f1a040cc920 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a041552c0 msgr2=0x7f1a040b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a041552c0 0x7f1a040b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 >> v1:172.21.15.70:6789/0 conn(0x7f1a040caec0 legacy=0x7f1a040d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0415b720 con 0x7f1a040cc920 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3718881788 0 0) 0x7f19f4001670 con 0x7f1a040cc920 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 >> v1:172.21.15.44:6789/0 conn(0x7f1a040cc920 legacy=0x7f1a040cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19ec0039e0 0x7f19ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a041552c0 0x7f19ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19ec004d40 0x7f19ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0415b440 con 0x7f19ec0039e0 2024-04-22T07:04:17.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec00b9c0 con 0x7f19ec004d40 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f19ec00bbc0 con 0x7f1a041552c0 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0a190640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19ec0039e0 0x7f19ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0998f640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a041552c0 0x7f19ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0a991640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19ec004d40 0x7f19ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0a190640 1 -- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a041552c0 msgr2=0x7f19ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0a190640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a041552c0 0x7f19ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0a190640 1 -- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19ec004d40 msgr2=0x7f19ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0a190640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19ec004d40 0x7f19ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0a190640 1 -- 172.21.15.44:0/2244883210 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec003520 con 0x7f19ec0039e0 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.931+0000 7f1a0998f640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1a041552c0 0x7f19ec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.932+0000 7f1a0a190640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19ec0039e0 0x7f19ec003dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009fc0 tx=0x7f19ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6cf33fc1f87056 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.932+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc039070 con 0x7f19ec0039e0 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.932+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19fc02cb40 con 0x7f19ec0039e0 2024-04-22T07:04:17.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.932+0000 7f1a0918e640 1 -- 172.21.15.44:0/2244883210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc009c10 con 0x7f19ec0039e0 2024-04-22T07:04:17.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.932+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/2244883210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19ec0039e0 msgr2=0x7f19ec003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:17.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.932+0000 7f1a0c3f4640 1 --2- 172.21.15.44:0/2244883210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f19ec0039e0 0x7f19ec003dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009fc0 tx=0x7f19ec005830 comp rx=0 tx=0).stop 2024-04-22T07:04:17.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.932+0000 7f1a0a991640 1 -- 172.21.15.44:0/2244883210 reap_dead start 2024-04-22T07:04:17.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.933+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/2244883210 shutdown_connections 2024-04-22T07:04:17.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.933+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/2244883210 >> 172.21.15.44:0/2244883210 conn(0x7f1a040676b0 msgr2=0x7f1a04067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:17.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.933+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/2244883210 shutdown_connections 2024-04-22T07:04:17.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.933+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/2244883210 wait complete. 2024-04-22T07:04:17.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.934+0000 7f1a0c3f4640 1 Processor -- start 2024-04-22T07:04:17.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.934+0000 7f1a0c3f4640 1 -- start start 2024-04-22T07:04:17.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.934+0000 7f1a0c3f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19ec0039e0 0x7f1a041584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:17.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.934+0000 7f1a0c3f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a0415c0d0 0x7f1a0414e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:17.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.934+0000 7f1a0c3f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a0415e760 0x7f1a0414edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:17.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.934+0000 7f1a0c3f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1a040c6a00 con 0x7f1a0415c0d0 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.934+0000 7f1a0c3f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0415bce0 con 0x7f1a0415e760 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.934+0000 7f1a0c3f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1a040d1130 con 0x7f19ec0039e0 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0998f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a0415c0d0 0x7f1a0414e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0998f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a0415c0d0 0x7f1a0414e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37778/0 (socket says 172.21.15.44:37778) 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0998f640 1 -- 172.21.15.44:0/3968959102 learned_addr learned my addr 172.21.15.44:0/3968959102 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0a991640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a0415e760 0x7f1a0414edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0a190640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19ec0039e0 0x7f1a041584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0998f640 1 -- 172.21.15.44:0/3968959102 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19ec0039e0 msgr2=0x7f1a041584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0998f640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19ec0039e0 0x7f1a041584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0998f640 1 -- 172.21.15.44:0/3968959102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a0415e760 msgr2=0x7f1a0414edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0998f640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a0415e760 0x7f1a0414edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0998f640 1 -- 172.21.15.44:0/3968959102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0415da10 con 0x7f1a0415c0d0 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0a991640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a0415e760 0x7f1a0414edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:17.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0998f640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a0415c0d0 0x7f1a0414e2c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19f800e850 tx=0x7f19f800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a02ffd640 1 -- 172.21.15.44:0/3968959102 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f800ee40 con 0x7f1a0415c0d0 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a02ffd640 1 -- 172.21.15.44:0/3968959102 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f8018070 con 0x7f1a0415c0d0 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/3968959102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a040b4dc0 con 0x7f1a0415c0d0 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a02ffd640 1 -- 172.21.15.44:0/3968959102 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f800c6c0 con 0x7f1a0415c0d0 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.935+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/3968959102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a041f18f0 con 0x7f1a0415c0d0 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.936+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/3968959102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a041556a0 con 0x7f1a0415c0d0 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.937+0000 7f1a02ffd640 1 -- 172.21.15.44:0/3968959102 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19f800c860 con 0x7f1a0415c0d0 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.937+0000 7f1a02ffd640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f19f006c8f0 0x7f19f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.937+0000 7f1a02ffd640 1 -- 172.21.15.44:0/3968959102 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f19f808b920 con 0x7f1a0415c0d0 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.937+0000 7f1a0a190640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f19f006c8f0 0x7f19f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:17.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.938+0000 7f1a0a190640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f19f006c8f0 0x7f19f006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f19fc036000 tx=0x7f19fc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:17.941 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:17.942+0000 7f1a02ffd640 1 -- 172.21.15.44:0/3968959102 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a041556a0 con 0x7f1a0415c0d0 2024-04-22T07:04:18.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.165+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/3968959102 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f1a0415a2f0 con 0x7f1a0415c0d0 2024-04-22T07:04:18.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.803+0000 7f1a02ffd640 1 -- 172.21.15.44:0/3968959102 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7f19f8059de0 con 0x7f1a0415c0d0 2024-04-22T07:04:18.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.805+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/3968959102 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f19f006c8f0 msgr2=0x7f19f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.805+0000 7f1a0c3f4640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f19f006c8f0 0x7f19f006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f19fc036000 tx=0x7f19fc02e000 comp rx=0 tx=0).stop 2024-04-22T07:04:18.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.805+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/3968959102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a0415c0d0 msgr2=0x7f1a0414e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.805+0000 7f1a0c3f4640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a0415c0d0 0x7f1a0414e2c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19f800e850 tx=0x7f19f800ed10 comp rx=0 tx=0).stop 2024-04-22T07:04:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.806+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/3968959102 shutdown_connections 2024-04-22T07:04:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.806+0000 7f1a0c3f4640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f19f006c8f0 0x7f19f006edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.806+0000 7f1a0c3f4640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a0415e760 0x7f1a0414edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.806+0000 7f1a0c3f4640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1a0415c0d0 0x7f1a0414e2c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.806+0000 7f1a0c3f4640 1 --2- 172.21.15.44:0/3968959102 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f19ec0039e0 0x7f1a041584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.806+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/3968959102 >> 172.21.15.44:0/3968959102 conn(0x7f1a040676b0 msgr2=0x7f1a040cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.806+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/3968959102 shutdown_connections 2024-04-22T07:04:18.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.806+0000 7f1a0c3f4640 1 -- 172.21.15.44:0/3968959102 wait complete. 2024-04-22T07:04:18.817 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-22T07:04:18.817 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.921+0000 7fa8061b6640 1 Processor -- start 2024-04-22T07:04:18.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.921+0000 7fa8061b6640 1 -- start start 2024-04-22T07:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.922+0000 7fa8061b6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000c4430 0x7fa8000c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.922+0000 7fa8061b6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8001564a0 0x7fa800159290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.922+0000 7fa8061b6640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8000c3970 con 0x7fa8001560c0 2024-04-22T07:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.922+0000 7fa8061b6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa800158e00 con 0x7fa8001564a0 2024-04-22T07:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.922+0000 7fa8061b6640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa8000c9f60 con 0x7fa8000c4430 2024-04-22T07:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.922+0000 7fa7ff7fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fa8001560c0 0x7fa8000c3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:37988/0 (socket says 172.21.15.44:37988) 2024-04-22T07:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.922+0000 7fa7ff7fe640 1 -- 172.21.15.44:0/986992379 learned_addr learned my addr 172.21.15.44:0/986992379 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.922+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810845636 0 0) 0x7fa8000c3970 con 0x7fa8001560c0 2024-04-22T07:04:18.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7e8003520 con 0x7fa8001560c0 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7feffd640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000c4430 0x7fa8000c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fffff640 1 --2- 172.21.15.44:0/986992379 >> v2:172.21.15.70:3300/0 conn(0x7fa8001564a0 0x7fa800159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2439900577 0 0) 0x7fa7e8003520 con 0x7fa8001560c0 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000c4430 msgr2=0x7fa8000c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa8000c4430 0x7fa8000c3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 >> v2:172.21.15.70:3300/0 conn(0x7fa8001564a0 msgr2=0x7fa800159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 --2- 172.21.15.44:0/986992379 >> v2:172.21.15.70:3300/0 conn(0x7fa8001564a0 0x7fa800159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8000ca240 con 0x7fa8001560c0 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3718881788 0 0) 0x7fa7f0001690 con 0x7fa8001560c0 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 >> v1:172.21.15.44:6789/0 conn(0x7fa8001560c0 legacy=0x7fa8000c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7e80039b0 0x7fa7e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e8004800 0x7fa7e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7e80051c0 0x7fa7e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa800158e00 con 0x7fa7e80039b0 2024-04-22T07:04:18.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000c9f60 con 0x7fa7e8004800 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e80045a0 con 0x7fa7e80051c0 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7ff7fe640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7e80039b0 0x7fa7e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7feffd640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e8004800 0x7fa7e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7fffff640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7e80051c0 0x7fa7e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7ff7fe640 1 -- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7e80051c0 msgr2=0x7fa7e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7ff7fe640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7e80051c0 0x7fa7e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7ff7fe640 1 -- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e8004800 msgr2=0x7fa7e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7ff7fe640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e8004800 0x7fa7e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.923+0000 7fa7ff7fe640 1 -- 172.21.15.44:0/986992379 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e8003520 con 0x7fa7e80039b0 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.924+0000 7fa7ff7fe640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7e80039b0 0x7fa7e8003dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c210 tx=0x7fa7e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea16518f165727c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.924+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0007d50 con 0x7fa7e80039b0 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.924+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f0030070 con 0x7fa7e80039b0 2024-04-22T07:04:18.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.924+0000 7fa7fe7fc640 1 -- 172.21.15.44:0/986992379 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0005d30 con 0x7fa7e80039b0 2024-04-22T07:04:18.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.925+0000 7fa8061b6640 1 -- 172.21.15.44:0/986992379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7e80039b0 msgr2=0x7fa7e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.925+0000 7fa8061b6640 1 --2- 172.21.15.44:0/986992379 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa7e80039b0 0x7fa7e8003dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c210 tx=0x7fa7e8005830 comp rx=0 tx=0).stop 2024-04-22T07:04:18.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.925+0000 7fa7fffff640 1 -- 172.21.15.44:0/986992379 reap_dead start 2024-04-22T07:04:18.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.925+0000 7fa8061b6640 1 -- 172.21.15.44:0/986992379 shutdown_connections 2024-04-22T07:04:18.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.925+0000 7fa8061b6640 1 -- 172.21.15.44:0/986992379 >> 172.21.15.44:0/986992379 conn(0x7fa8000713b0 msgr2=0x7fa8000717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.925+0000 7fa8061b6640 1 -- 172.21.15.44:0/986992379 shutdown_connections 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.925+0000 7fa8061b6640 1 -- 172.21.15.44:0/986992379 wait complete. 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.926+0000 7fa8061b6640 1 Processor -- start 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.926+0000 7fa8061b6640 1 -- start start 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa8061b6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7e80039b0 0x7fa8001e5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa8061b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000c81c0 0x7fa8001df420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa8061b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8001df960 0x7fa8001e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa8061b6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000caae0 con 0x7fa8001df960 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa8061b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa8000ca800 con 0x7fa8000c81c0 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa8061b6640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa8000cf080 con 0x7fa7e80039b0 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8001df960 0x7fa8001e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7fffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8001df960 0x7fa8001e3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37792/0 (socket says 172.21.15.44:37792) 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7fffff640 1 -- 172.21.15.44:0/846988907 learned_addr learned my addr 172.21.15.44:0/846988907 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7feffd640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000c81c0 0x7fa8001df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7ff7fe640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7e80039b0 0x7fa8001e5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7fffff640 1 -- 172.21.15.44:0/846988907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7e80039b0 msgr2=0x7fa8001e5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7fffff640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7e80039b0 0x7fa8001e5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7fffff640 1 -- 172.21.15.44:0/846988907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000c81c0 msgr2=0x7fa8001df420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7fffff640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000c81c0 0x7fa8001df420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7fffff640 1 -- 172.21.15.44:0/846988907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8001ef290 con 0x7fa8001df960 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.927+0000 7fa7fffff640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8001df960 0x7fa8001e3ca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8000c5730 tx=0x7fa7f800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.928+0000 7fa7e7fff640 1 -- 172.21.15.44:0/846988907 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8003e60 con 0x7fa8001df960 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.928+0000 7fa7e7fff640 1 -- 172.21.15.44:0/846988907 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f801c070 con 0x7fa8001df960 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.928+0000 7fa8061b6640 1 -- 172.21.15.44:0/846988907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8000c5940 con 0x7fa8001df960 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.928+0000 7fa7e7fff640 1 -- 172.21.15.44:0/846988907 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800fb70 con 0x7fa8001df960 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.928+0000 7fa8061b6640 1 -- 172.21.15.44:0/846988907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8000c5f00 con 0x7fa8001df960 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.929+0000 7fa8061b6640 1 -- 172.21.15.44:0/846988907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8000c8a50 con 0x7fa8001df960 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.929+0000 7fa7e7fff640 1 -- 172.21.15.44:0/846988907 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7f800fd10 con 0x7fa8001df960 2024-04-22T07:04:18.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.929+0000 7fa7e7fff640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa7ec06c8f0 0x7fa7ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:18.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.930+0000 7fa7e7fff640 1 -- 172.21.15.44:0/846988907 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa7f808c370 con 0x7fa8001df960 2024-04-22T07:04:18.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.930+0000 7fa7ff7fe640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa7ec06c8f0 0x7fa7ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:18.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.930+0000 7fa7ff7fe640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa7ec06c8f0 0x7fa7ec06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c210 tx=0x7fa7f0007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:18.934+0000 7fa7e7fff640 1 -- 172.21.15.44:0/846988907 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8000c8a50 con 0x7fa8001df960 2024-04-22T07:04:19.161 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.163+0000 7fa8061b6640 1 -- 172.21.15.44:0/846988907 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa800069ea0 con 0x7fa8001df960 2024-04-22T07:04:19.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.163+0000 7fa7e7fff640 1 -- 172.21.15.44:0/846988907 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6819 (secure 0 0 0) 0x7fa7f8010040 con 0x7fa8001df960 2024-04-22T07:04:19.162 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:19.162 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4303,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:04:18.167616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0],"up":{"mds_0":4324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-22T07:04:19.163 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 8 2024-04-22T07:04:19.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.165+0000 7fa8061b6640 1 -- 172.21.15.44:0/846988907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa7ec06c8f0 msgr2=0x7fa7ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.165+0000 7fa8061b6640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa7ec06c8f0 0x7fa7ec06edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c210 tx=0x7fa7f0007000 comp rx=0 tx=0).stop 2024-04-22T07:04:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.165+0000 7fa8061b6640 1 -- 172.21.15.44:0/846988907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8001df960 msgr2=0x7fa8001e3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.165+0000 7fa8061b6640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8001df960 0x7fa8001e3ca0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8000c5730 tx=0x7fa7f800f1b0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.166+0000 7fa8061b6640 1 -- 172.21.15.44:0/846988907 shutdown_connections 2024-04-22T07:04:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.166+0000 7fa8061b6640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa7ec06c8f0 0x7fa7ec06edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.166+0000 7fa8061b6640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa8001df960 0x7fa8001e3ca0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.166+0000 7fa8061b6640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8000c81c0 0x7fa8001df420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.166+0000 7fa8061b6640 1 --2- 172.21.15.44:0/846988907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa7e80039b0 0x7fa8001e5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.166+0000 7fa8061b6640 1 -- 172.21.15.44:0/846988907 >> 172.21.15.44:0/846988907 conn(0x7fa8000713b0 msgr2=0x7fa800159290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.166+0000 7fa8061b6640 1 -- 172.21.15.44:0/846988907 shutdown_connections 2024-04-22T07:04:19.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.166+0000 7fa8061b6640 1 -- 172.21.15.44:0/846988907 wait complete. 2024-04-22T07:04:19.175 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:19.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.277+0000 7f766e669640 1 Processor -- start 2024-04-22T07:04:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.277+0000 7f766e669640 1 -- start start 2024-04-22T07:04:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.277+0000 7f766e669640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7668154f70 0x7f766815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.277+0000 7f766e669640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f76680cb3e0 0x7f76680cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.277+0000 7f766e669640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f766815ee90 con 0x7f76680cd6f0 2024-04-22T07:04:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.277+0000 7f766e669640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f76680cd260 con 0x7f76680cb3e0 2024-04-22T07:04:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.277+0000 7f766e669640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f766815f170 con 0x7f7668154f70 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.278+0000 7f766cc06640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f76680cb3e0 0x7f76680cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.278+0000 7f766cc06640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f76680cb3e0 0x7f76680cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37802/0 (socket says 172.21.15.44:37802) 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.278+0000 7f766cc06640 1 -- 172.21.15.44:0/3878254504 learned_addr learned my addr 172.21.15.44:0/3878254504 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.278+0000 7f7667fff640 1 --2- 172.21.15.44:0/3878254504 >> v2:172.21.15.70:3300/0 conn(0x7f7668154f70 0x7f766815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.278+0000 7f766cc06640 1 -- 172.21.15.44:0/3878254504 >> v2:172.21.15.70:3300/0 conn(0x7f7668154f70 msgr2=0x7f766815ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.278+0000 7f766cc06640 1 --2- 172.21.15.44:0/3878254504 >> v2:172.21.15.70:3300/0 conn(0x7f7668154f70 0x7f766815ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.278+0000 7f766cc06640 1 -- 172.21.15.44:0/3878254504 >> v1:172.21.15.70:6789/0 conn(0x7f76680cd6f0 legacy=0x7f76680c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.278+0000 7f766cc06640 1 -- 172.21.15.44:0/3878254504 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76680cc930 con 0x7f76680cb3e0 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f766cc06640 1 --2- 172.21.15.44:0/3878254504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cb3e0 0x7f76680cb850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76680cafe0 tx=0x7f766002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8501116c021bcb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f7666ffd640 1 -- 172.21.15.44:0/3878254504 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766002cbb0 con 0x7f76680cb3e0 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f7666ffd640 1 -- 172.21.15.44:0/3878254504 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766002cd50 con 0x7f76680cb3e0 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f7666ffd640 1 -- 172.21.15.44:0/3878254504 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660034610 con 0x7f76680cb3e0 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f766e669640 1 -- 172.21.15.44:0/3878254504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cb3e0 msgr2=0x7f76680cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f766e669640 1 --2- 172.21.15.44:0/3878254504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cb3e0 0x7f76680cb850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76680cafe0 tx=0x7f766002c010 comp rx=0 tx=0).stop 2024-04-22T07:04:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f766e669640 1 -- 172.21.15.44:0/3878254504 shutdown_connections 2024-04-22T07:04:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f766e669640 1 --2- 172.21.15.44:0/3878254504 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cb3e0 0x7f76680cb850 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f766e669640 1 --2- 172.21.15.44:0/3878254504 >> v2:172.21.15.70:3300/0 conn(0x7f7668154f70 0x7f766815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f766e669640 1 -- 172.21.15.44:0/3878254504 >> 172.21.15.44:0/3878254504 conn(0x7f7668067d40 msgr2=0x7f766806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f766e669640 1 -- 172.21.15.44:0/3878254504 shutdown_connections 2024-04-22T07:04:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.279+0000 7f766e669640 1 -- 172.21.15.44:0/3878254504 wait complete. 2024-04-22T07:04:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.280+0000 7f766e669640 1 Processor -- start 2024-04-22T07:04:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.281+0000 7f766e669640 1 -- start start 2024-04-22T07:04:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.281+0000 7f766e669640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680cb3e0 0x7f766814add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.281+0000 7f766e669640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cd6f0 0x7f7668159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.281+0000 7f766e669640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7668154f70 0x7f766815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.281+0000 7f766e669640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7668155350 con 0x7f76680cd6f0 2024-04-22T07:04:19.282 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.281+0000 7f766e669640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f76680ccef0 con 0x7f76680cb3e0 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.281+0000 7f766e669640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f766814fe20 con 0x7f7668154f70 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.281+0000 7f76677fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cd6f0 0x7f7668159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.281+0000 7f76677fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cd6f0 0x7f7668159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37814/0 (socket says 172.21.15.44:37814) 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.281+0000 7f76677fe640 1 -- 172.21.15.44:0/3827766801 learned_addr learned my addr 172.21.15.44:0/3827766801 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f766cc06640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7668154f70 0x7f766815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f7667fff640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680cb3e0 0x7f766814add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f76677fe640 1 -- 172.21.15.44:0/3827766801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7668154f70 msgr2=0x7f766815a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f76677fe640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7668154f70 0x7f766815a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f76677fe640 1 -- 172.21.15.44:0/3827766801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680cb3e0 msgr2=0x7f766814add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f76677fe640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680cb3e0 0x7f766814add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f76677fe640 1 -- 172.21.15.44:0/3827766801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766814b310 con 0x7f76680cd6f0 2024-04-22T07:04:19.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f76677fe640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cd6f0 0x7f7668159d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f765c00c400 tx=0x7f765c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f7664ff9640 1 -- 172.21.15.44:0/3827766801 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c0052d0 con 0x7f76680cd6f0 2024-04-22T07:04:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f7664ff9640 1 -- 172.21.15.44:0/3827766801 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f765c005490 con 0x7f76680cd6f0 2024-04-22T07:04:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f766e669640 1 -- 172.21.15.44:0/3827766801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7668151120 con 0x7f76680cd6f0 2024-04-22T07:04:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f7664ff9640 1 -- 172.21.15.44:0/3827766801 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c007260 con 0x7f76680cd6f0 2024-04-22T07:04:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.282+0000 7f766e669640 1 -- 172.21.15.44:0/3827766801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f766814f4f0 con 0x7f76680cd6f0 2024-04-22T07:04:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.283+0000 7f766e669640 1 -- 172.21.15.44:0/3827766801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76680cc500 con 0x7f76680cd6f0 2024-04-22T07:04:19.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.284+0000 7f7664ff9640 1 -- 172.21.15.44:0/3827766801 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f765c007400 con 0x7f76680cd6f0 2024-04-22T07:04:19.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.284+0000 7f7664ff9640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f765406c8f0 0x7f765406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.284+0000 7f7664ff9640 1 -- 172.21.15.44:0/3827766801 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f765c08b160 con 0x7f76680cd6f0 2024-04-22T07:04:19.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.284+0000 7f7667fff640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f765406c8f0 0x7f765406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.285 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.285+0000 7f7667fff640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f765406c8f0 0x7f765406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7658006fd0 tx=0x7f7658008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:19.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.289+0000 7f7664ff9640 1 -- 172.21.15.44:0/3827766801 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f765c059500 con 0x7f76680cd6f0 2024-04-22T07:04:19.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.476+0000 7f766e669640 1 -- 172.21.15.44:0/3827766801 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f766804fdc0 con 0x7f76680cd6f0 2024-04-22T07:04:19.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.477+0000 7f7664ff9640 1 -- 172.21.15.44:0/3827766801 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16557 (secure 0 0 0) 0x7f765c058e00 con 0x7f76680cd6f0 2024-04-22T07:04:19.476 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:19.476 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":23,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:14.778262+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 -- 172.21.15.44:0/3827766801 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f765406c8f0 msgr2=0x7f765406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f765406c8f0 0x7f765406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7658006fd0 tx=0x7f7658008040 comp rx=0 tx=0).stop 2024-04-22T07:04:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 -- 172.21.15.44:0/3827766801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cd6f0 msgr2=0x7f7668159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cd6f0 0x7f7668159d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f765c00c400 tx=0x7f765c00c8c0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 -- 172.21.15.44:0/3827766801 shutdown_connections 2024-04-22T07:04:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f765406c8f0 0x7f765406edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7668154f70 0x7f766815a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76680cd6f0 0x7f7668159d00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 --2- 172.21.15.44:0/3827766801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76680cb3e0 0x7f766814add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 -- 172.21.15.44:0/3827766801 >> 172.21.15.44:0/3827766801 conn(0x7f7668067d40 msgr2=0x7f766815bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 -- 172.21.15.44:0/3827766801 shutdown_connections 2024-04-22T07:04:19.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.479+0000 7f766e669640 1 -- 172.21.15.44:0/3827766801 wait complete. 2024-04-22T07:04:19.489 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-22T07:04:19.489 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:19.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.590+0000 7fc3bad0d640 1 Processor -- start 2024-04-22T07:04:19.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.590+0000 7fc3bad0d640 1 -- start start 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.591+0000 7fc3bad0d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 0x7fc3b406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.591+0000 7fc3bad0d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc3b40c8d60 0x7fc3b40ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.591+0000 7fc3bad0d640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b415ae60 con 0x7fc3b40ce850 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.591+0000 7fc3bad0d640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b4155600 con 0x7fc3b40c8d60 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.591+0000 7fc3bad0d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b40c9ee0 con 0x7fc3b4155220 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.591+0000 7fc3b3fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc3b40c8d60 0x7fc3b40ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.591+0000 7fc3b92aa640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fc3b40ce850 0x7fc3b415b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38000/0 (socket says 172.21.15.44:38000) 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.591+0000 7fc3b3fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc3b40c8d60 0x7fc3b40ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37826/0 (socket says 172.21.15.44:37826) 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.591+0000 7fc3b92aa640 1 -- 172.21.15.44:0/4236733392 learned_addr learned my addr 172.21.15.44:0/4236733392 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.592+0000 7fc3b8aa9640 1 --2- 172.21.15.44:0/4236733392 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 0x7fc3b406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.592+0000 7fc3b3fff640 1 -- 172.21.15.44:0/4236733392 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 msgr2=0x7fc3b406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.592+0000 7fc3b3fff640 1 --2- 172.21.15.44:0/4236733392 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 0x7fc3b406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.592+0000 7fc3b3fff640 1 -- 172.21.15.44:0/4236733392 >> v1:172.21.15.44:6789/0 conn(0x7fc3b40ce850 legacy=0x7fc3b415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.592+0000 7fc3b3fff640 1 -- 172.21.15.44:0/4236733392 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b40ca1c0 con 0x7fc3b40c8d60 2024-04-22T07:04:19.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.592+0000 7fc3b37fe640 1 -- 172.21.15.44:0/4236733392 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480714765 0 0) 0x7fc3b415ae60 con 0x7fc3b40ce850 2024-04-22T07:04:19.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.592+0000 7fc3b8aa9640 1 --2- 172.21.15.44:0/4236733392 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 0x7fc3b406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:19.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.592+0000 7fc3b3fff640 1 --2- 172.21.15.44:0/4236733392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40c8d60 0x7fc3b40ce310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40097a0 tx=0x7fc3a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81163e37b2703d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:19.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3b37fe640 1 -- 172.21.15.44:0/4236733392 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4035020 con 0x7fc3b40c8d60 2024-04-22T07:04:19.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3b37fe640 1 -- 172.21.15.44:0/4236733392 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a4031830 con 0x7fc3b40c8d60 2024-04-22T07:04:19.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3b37fe640 1 -- 172.21.15.44:0/4236733392 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4031c00 con 0x7fc3b40c8d60 2024-04-22T07:04:19.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3bad0d640 1 -- 172.21.15.44:0/4236733392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40c8d60 msgr2=0x7fc3b40ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3bad0d640 1 --2- 172.21.15.44:0/4236733392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40c8d60 0x7fc3b40ce310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40097a0 tx=0x7fc3a402b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3bad0d640 1 -- 172.21.15.44:0/4236733392 shutdown_connections 2024-04-22T07:04:19.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3bad0d640 1 --2- 172.21.15.44:0/4236733392 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40c8d60 0x7fc3b40ce310 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3bad0d640 1 --2- 172.21.15.44:0/4236733392 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 0x7fc3b406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3bad0d640 1 -- 172.21.15.44:0/4236733392 >> 172.21.15.44:0/4236733392 conn(0x7fc3b40713b0 msgr2=0x7fc3b40717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:19.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3bad0d640 1 -- 172.21.15.44:0/4236733392 shutdown_connections 2024-04-22T07:04:19.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.593+0000 7fc3bad0d640 1 -- 172.21.15.44:0/4236733392 wait complete. 2024-04-22T07:04:19.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.594+0000 7fc3bad0d640 1 Processor -- start 2024-04-22T07:04:19.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.594+0000 7fc3bad0d640 1 -- start start 2024-04-22T07:04:19.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.595+0000 7fc3bad0d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3b40c8d60 0x7fc3b40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.595+0000 7fc3bad0d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40ce850 0x7fc3b41508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.595+0000 7fc3bad0d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 0x7fc3b4150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.595+0000 7fc3bad0d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b40caa60 con 0x7fc3b40ce850 2024-04-22T07:04:19.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.595+0000 7fc3bad0d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b40ca780 con 0x7fc3b40c8d60 2024-04-22T07:04:19.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.595+0000 7fc3bad0d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b406beb0 con 0x7fc3b4155220 2024-04-22T07:04:19.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.595+0000 7fc3b3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40ce850 0x7fc3b41508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.595+0000 7fc3b3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40ce850 0x7fc3b41508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37840/0 (socket says 172.21.15.44:37840) 2024-04-22T07:04:19.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.595+0000 7fc3b3fff640 1 -- 172.21.15.44:0/3017123190 learned_addr learned my addr 172.21.15.44:0/3017123190 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3b3fff640 1 -- 172.21.15.44:0/3017123190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 msgr2=0x7fc3b4150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3b92aa640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 0x7fc3b4150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3b3fff640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 0x7fc3b4150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3b3fff640 1 -- 172.21.15.44:0/3017123190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3b40c8d60 msgr2=0x7fc3b40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3b3fff640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3b40c8d60 0x7fc3b40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3b3fff640 1 -- 172.21.15.44:0/3017123190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b41f4840 con 0x7fc3b40ce850 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3b3fff640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40ce850 0x7fc3b41508b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009890 tx=0x7fc3a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3b17fa640 1 -- 172.21.15.44:0/3017123190 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4041070 con 0x7fc3b40ce850 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3b17fa640 1 -- 172.21.15.44:0/3017123190 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a403b920 con 0x7fc3b40ce850 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3bad0d640 1 -- 172.21.15.44:0/3017123190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b41f4b20 con 0x7fc3b40ce850 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3b17fa640 1 -- 172.21.15.44:0/3017123190 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4035040 con 0x7fc3b40ce850 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.596+0000 7fc3bad0d640 1 -- 172.21.15.44:0/3017123190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b41500b0 con 0x7fc3b40ce850 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.597+0000 7fc3bad0d640 1 -- 172.21.15.44:0/3017123190 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b4069ea0 con 0x7fc3b40ce850 2024-04-22T07:04:19.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.597+0000 7fc3b17fa640 1 -- 172.21.15.44:0/3017123190 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc3a404c060 con 0x7fc3b40ce850 2024-04-22T07:04:19.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.598+0000 7fc3b17fa640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc39406c8f0 0x7fc39406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.598+0000 7fc3b17fa640 1 -- 172.21.15.44:0/3017123190 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc3a40ad520 con 0x7fc3b40ce850 2024-04-22T07:04:19.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.598+0000 7fc3b8aa9640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc39406c8f0 0x7fc39406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.598+0000 7fc3b8aa9640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc39406c8f0 0x7fc39406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009eb0 tx=0x7fc3ac0094a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.602+0000 7fc3b17fa640 1 -- 172.21.15.44:0/3017123190 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3a407b8d0 con 0x7fc3b40ce850 2024-04-22T07:04:19.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.783+0000 7fc3bad0d640 1 -- 172.21.15.44:0/3017123190 --> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3b404fdc0 con 0x7fc39406c8f0 2024-04-22T07:04:19.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.790+0000 7fc3b17fa640 1 -- 172.21.15.44:0/3017123190 <== mgr.4101 v2:172.21.15.44:6832/1889438723 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251849 (secure 0 0 0) 0x7fc3b404fdc0 con 0x7fc39406c8f0 2024-04-22T07:04:19.792 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:19.793 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T07:04:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 -- 172.21.15.44:0/3017123190 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc39406c8f0 msgr2=0x7fc39406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc39406c8f0 0x7fc39406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009eb0 tx=0x7fc3ac0094a0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 -- 172.21.15.44:0/3017123190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40ce850 msgr2=0x7fc3b41508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40ce850 0x7fc3b41508b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009890 tx=0x7fc3a40046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 -- 172.21.15.44:0/3017123190 shutdown_connections 2024-04-22T07:04:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc39406c8f0 0x7fc39406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc3b4155220 0x7fc3b4150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc3b40ce850 0x7fc3b41508b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 --2- 172.21.15.44:0/3017123190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3b40c8d60 0x7fc3b40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 -- 172.21.15.44:0/3017123190 >> 172.21.15.44:0/3017123190 conn(0x7fc3b40713b0 msgr2=0x7fc3b40c49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 -- 172.21.15.44:0/3017123190 shutdown_connections 2024-04-22T07:04:19.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.793+0000 7fc3bad0d640 1 -- 172.21.15.44:0/3017123190 wait complete. 2024-04-22T07:04:19.802 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-22T07:04:19.711518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239940,"kb_used_data":2972,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734780,"statfs":{"total":773094113280,"available":772848414720,"internally_reserved":0,"allocated":3043328,"data_stored":1824316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.941791"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961498+0000","last_change":"2024-04-22T07:04:13.800864+0000","last_active":"2024-04-22T07:04:14.961498+0000","last_peered":"2024-04-22T07:04:14.961498+0000","last_clean":"2024-04-22T07:04:14.961498+0000","last_became_active":"2024-04-22T07:04:13.800611+0000","last_became_peered":"2024-04-22T07:04:13.800611+0000","last_unstale":"2024-04-22T07:04:14.961498+0000","last_undegraded":"2024-04-22T07:04:14.961498+0000","last_fullsized":"2024-04-22T07:04:14.961498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:55.129127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056077+0000","last_change":"2024-04-22T07:04:13.798390+0000","last_active":"2024-04-22T07:04:15.056077+0000","last_peered":"2024-04-22T07:04:15.056077+0000","last_clean":"2024-04-22T07:04:15.056077+0000","last_became_active":"2024-04-22T07:04:13.797927+0000","last_became_peered":"2024-04-22T07:04:13.797927+0000","last_unstale":"2024-04-22T07:04:15.056077+0000","last_undegraded":"2024-04-22T07:04:15.056077+0000","last_fullsized":"2024-04-22T07:04:15.056077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056425+0000","last_change":"2024-04-22T07:04:13.795288+0000","last_active":"2024-04-22T07:04:15.056425+0000","last_peered":"2024-04-22T07:04:15.056425+0000","last_clean":"2024-04-22T07:04:15.056425+0000","last_became_active":"2024-04-22T07:04:13.794704+0000","last_became_peered":"2024-04-22T07:04:13.794704+0000","last_unstale":"2024-04-22T07:04:15.056425+0000","last_undegraded":"2024-04-22T07:04:15.056425+0000","last_fullsized":"2024-04-22T07:04:15.056425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783223+0000","last_change":"2024-04-22T07:04:13.801884+0000","last_active":"2024-04-22T07:04:14.783223+0000","last_peered":"2024-04-22T07:04:14.783223+0000","last_clean":"2024-04-22T07:04:14.783223+0000","last_became_active":"2024-04-22T07:04:13.801620+0000","last_became_peered":"2024-04-22T07:04:13.801620+0000","last_unstale":"2024-04-22T07:04:14.783223+0000","last_undegraded":"2024-04-22T07:04:14.783223+0000","last_fullsized":"2024-04-22T07:04:14.783223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961151+0000","last_change":"2024-04-22T07:04:13.798670+0000","last_active":"2024-04-22T07:04:14.961151+0000","last_peered":"2024-04-22T07:04:14.961151+0000","last_clean":"2024-04-22T07:04:14.961151+0000","last_became_active":"2024-04-22T07:04:13.798384+0000","last_became_peered":"2024-04-22T07:04:13.798384+0000","last_unstale":"2024-04-22T07:04:14.961151+0000","last_undegraded":"2024-04-22T07:04:14.961151+0000","last_fullsized":"2024-04-22T07:04:14.961151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783124+0000","last_change":"2024-04-22T07:04:13.796468+0000","last_active":"2024-04-22T07:04:14.783124+0000","last_peered":"2024-04-22T07:04:14.783124+0000","last_clean":"2024-04-22T07:04:14.783124+0000","last_became_active":"2024-04-22T07:04:13.796139+0000","last_became_peered":"2024-04-22T07:04:13.796139+0000","last_unstale":"2024-04-22T07:04:14.783124+0000","last_undegraded":"2024-04-22T07:04:14.783124+0000","last_fullsized":"2024-04-22T07:04:14.783124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961066+0000","last_change":"2024-04-22T07:04:13.801449+0000","last_active":"2024-04-22T07:04:14.961066+0000","last_peered":"2024-04-22T07:04:14.961066+0000","last_clean":"2024-04-22T07:04:14.961066+0000","last_became_active":"2024-04-22T07:04:13.799374+0000","last_became_peered":"2024-04-22T07:04:13.799374+0000","last_unstale":"2024-04-22T07:04:14.961066+0000","last_undegraded":"2024-04-22T07:04:14.961066+0000","last_fullsized":"2024-04-22T07:04:14.961066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054559+0000","last_change":"2024-04-22T07:04:13.802780+0000","last_active":"2024-04-22T07:04:15.054559+0000","last_peered":"2024-04-22T07:04:15.054559+0000","last_clean":"2024-04-22T07:04:15.054559+0000","last_became_active":"2024-04-22T07:04:13.802371+0000","last_became_peered":"2024-04-22T07:04:13.802371+0000","last_unstale":"2024-04-22T07:04:15.054559+0000","last_undegraded":"2024-04-22T07:04:15.054559+0000","last_fullsized":"2024-04-22T07:04:15.054559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961532+0000","last_change":"2024-04-22T07:04:13.792483+0000","last_active":"2024-04-22T07:04:14.961532+0000","last_peered":"2024-04-22T07:04:14.961532+0000","last_clean":"2024-04-22T07:04:14.961532+0000","last_became_active":"2024-04-22T07:04:13.792198+0000","last_became_peered":"2024-04-22T07:04:13.792198+0000","last_unstale":"2024-04-22T07:04:14.961532+0000","last_undegraded":"2024-04-22T07:04:14.961532+0000","last_fullsized":"2024-04-22T07:04:14.961532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:36.943428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782040+0000","last_change":"2024-04-22T07:04:13.797379+0000","last_active":"2024-04-22T07:04:14.782040+0000","last_peered":"2024-04-22T07:04:14.782040+0000","last_clean":"2024-04-22T07:04:14.782040+0000","last_became_active":"2024-04-22T07:04:13.797029+0000","last_became_peered":"2024-04-22T07:04:13.797029+0000","last_unstale":"2024-04-22T07:04:14.782040+0000","last_undegraded":"2024-04-22T07:04:14.782040+0000","last_fullsized":"2024-04-22T07:04:14.782040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781680+0000","last_change":"2024-04-22T07:04:13.789875+0000","last_active":"2024-04-22T07:04:14.781680+0000","last_peered":"2024-04-22T07:04:14.781680+0000","last_clean":"2024-04-22T07:04:14.781680+0000","last_became_active":"2024-04-22T07:04:13.789598+0000","last_became_peered":"2024-04-22T07:04:13.789598+0000","last_unstale":"2024-04-22T07:04:14.781680+0000","last_undegraded":"2024-04-22T07:04:14.781680+0000","last_fullsized":"2024-04-22T07:04:14.781680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055156+0000","last_change":"2024-04-22T07:04:13.791103+0000","last_active":"2024-04-22T07:04:15.055156+0000","last_peered":"2024-04-22T07:04:15.055156+0000","last_clean":"2024-04-22T07:04:15.055156+0000","last_became_active":"2024-04-22T07:04:13.790819+0000","last_became_peered":"2024-04-22T07:04:13.790819+0000","last_unstale":"2024-04-22T07:04:15.055156+0000","last_undegraded":"2024-04-22T07:04:15.055156+0000","last_fullsized":"2024-04-22T07:04:15.055156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:50.632020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055402+0000","last_change":"2024-04-22T07:04:13.802407+0000","last_active":"2024-04-22T07:04:15.055402+0000","last_peered":"2024-04-22T07:04:15.055402+0000","last_clean":"2024-04-22T07:04:15.055402+0000","last_became_active":"2024-04-22T07:04:13.801785+0000","last_became_peered":"2024-04-22T07:04:13.801785+0000","last_unstale":"2024-04-22T07:04:15.055402+0000","last_undegraded":"2024-04-22T07:04:15.055402+0000","last_fullsized":"2024-04-22T07:04:15.055402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055557+0000","last_change":"2024-04-22T07:04:13.795888+0000","last_active":"2024-04-22T07:04:15.055557+0000","last_peered":"2024-04-22T07:04:15.055557+0000","last_clean":"2024-04-22T07:04:15.055557+0000","last_became_active":"2024-04-22T07:04:13.795092+0000","last_became_peered":"2024-04-22T07:04:13.795092+0000","last_unstale":"2024-04-22T07:04:15.055557+0000","last_undegraded":"2024-04-22T07:04:15.055557+0000","last_fullsized":"2024-04-22T07:04:15.055557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:54.119208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055926+0000","last_change":"2024-04-22T07:04:13.797138+0000","last_active":"2024-04-22T07:04:15.055926+0000","last_peered":"2024-04-22T07:04:15.055926+0000","last_clean":"2024-04-22T07:04:15.055926+0000","last_became_active":"2024-04-22T07:04:13.796763+0000","last_became_peered":"2024-04-22T07:04:13.796763+0000","last_unstale":"2024-04-22T07:04:15.055926+0000","last_undegraded":"2024-04-22T07:04:15.055926+0000","last_fullsized":"2024-04-22T07:04:15.055926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782465+0000","last_change":"2024-04-22T07:04:13.803869+0000","last_active":"2024-04-22T07:04:14.782465+0000","last_peered":"2024-04-22T07:04:14.782465+0000","last_clean":"2024-04-22T07:04:14.782465+0000","last_became_active":"2024-04-22T07:04:13.803451+0000","last_became_peered":"2024-04-22T07:04:13.803451+0000","last_unstale":"2024-04-22T07:04:14.782465+0000","last_undegraded":"2024-04-22T07:04:14.782465+0000","last_fullsized":"2024-04-22T07:04:14.782465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783077+0000","last_change":"2024-04-22T07:04:13.795746+0000","last_active":"2024-04-22T07:04:14.783077+0000","last_peered":"2024-04-22T07:04:14.783077+0000","last_clean":"2024-04-22T07:04:14.783077+0000","last_became_active":"2024-04-22T07:04:13.794694+0000","last_became_peered":"2024-04-22T07:04:13.794694+0000","last_unstale":"2024-04-22T07:04:14.783077+0000","last_undegraded":"2024-04-22T07:04:14.783077+0000","last_fullsized":"2024-04-22T07:04:14.783077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054829+0000","last_change":"2024-04-22T07:04:13.790670+0000","last_active":"2024-04-22T07:04:15.054829+0000","last_peered":"2024-04-22T07:04:15.054829+0000","last_clean":"2024-04-22T07:04:15.054829+0000","last_became_active":"2024-04-22T07:04:13.790104+0000","last_became_peered":"2024-04-22T07:04:13.790104+0000","last_unstale":"2024-04-22T07:04:15.054829+0000","last_undegraded":"2024-04-22T07:04:15.054829+0000","last_fullsized":"2024-04-22T07:04:15.054829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:09.325971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961464+0000","last_change":"2024-04-22T07:04:13.799016+0000","last_active":"2024-04-22T07:04:14.961464+0000","last_peered":"2024-04-22T07:04:14.961464+0000","last_clean":"2024-04-22T07:04:14.961464+0000","last_became_active":"2024-04-22T07:04:13.798716+0000","last_became_peered":"2024-04-22T07:04:13.798716+0000","last_unstale":"2024-04-22T07:04:14.961464+0000","last_undegraded":"2024-04-22T07:04:14.961464+0000","last_fullsized":"2024-04-22T07:04:14.961464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:36.609382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781586+0000","last_change":"2024-04-22T07:04:13.797296+0000","last_active":"2024-04-22T07:04:14.781586+0000","last_peered":"2024-04-22T07:04:14.781586+0000","last_clean":"2024-04-22T07:04:14.781586+0000","last_became_active":"2024-04-22T07:04:13.796997+0000","last_became_peered":"2024-04-22T07:04:13.796997+0000","last_unstale":"2024-04-22T07:04:14.781586+0000","last_undegraded":"2024-04-22T07:04:14.781586+0000","last_fullsized":"2024-04-22T07:04:14.781586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055994+0000","last_change":"2024-04-22T07:04:13.799675+0000","last_active":"2024-04-22T07:04:15.055994+0000","last_peered":"2024-04-22T07:04:15.055994+0000","last_clean":"2024-04-22T07:04:15.055994+0000","last_became_active":"2024-04-22T07:04:13.799411+0000","last_became_peered":"2024-04-22T07:04:13.799411+0000","last_unstale":"2024-04-22T07:04:15.055994+0000","last_undegraded":"2024-04-22T07:04:15.055994+0000","last_fullsized":"2024-04-22T07:04:15.055994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783058+0000","last_change":"2024-04-22T07:04:13.796128+0000","last_active":"2024-04-22T07:04:14.783058+0000","last_peered":"2024-04-22T07:04:14.783058+0000","last_clean":"2024-04-22T07:04:14.783058+0000","last_became_active":"2024-04-22T07:04:13.795330+0000","last_became_peered":"2024-04-22T07:04:13.795330+0000","last_unstale":"2024-04-22T07:04:14.783058+0000","last_undegraded":"2024-04-22T07:04:14.783058+0000","last_fullsized":"2024-04-22T07:04:14.783058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055294+0000","last_change":"2024-04-22T07:04:13.800458+0000","last_active":"2024-04-22T07:04:15.055294+0000","last_peered":"2024-04-22T07:04:15.055294+0000","last_clean":"2024-04-22T07:04:15.055294+0000","last_became_active":"2024-04-22T07:04:13.800171+0000","last_became_peered":"2024-04-22T07:04:13.800171+0000","last_unstale":"2024-04-22T07:04:15.055294+0000","last_undegraded":"2024-04-22T07:04:15.055294+0000","last_fullsized":"2024-04-22T07:04:15.055294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782518+0000","last_change":"2024-04-22T07:04:13.802472+0000","last_active":"2024-04-22T07:04:14.782518+0000","last_peered":"2024-04-22T07:04:14.782518+0000","last_clean":"2024-04-22T07:04:14.782518+0000","last_became_active":"2024-04-22T07:04:13.802201+0000","last_became_peered":"2024-04-22T07:04:13.802201+0000","last_unstale":"2024-04-22T07:04:14.782518+0000","last_undegraded":"2024-04-22T07:04:14.782518+0000","last_fullsized":"2024-04-22T07:04:14.782518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055335+0000","last_change":"2024-04-22T07:04:13.795055+0000","last_active":"2024-04-22T07:04:15.055335+0000","last_peered":"2024-04-22T07:04:15.055335+0000","last_clean":"2024-04-22T07:04:15.055335+0000","last_became_active":"2024-04-22T07:04:13.794145+0000","last_became_peered":"2024-04-22T07:04:13.794145+0000","last_unstale":"2024-04-22T07:04:15.055335+0000","last_undegraded":"2024-04-22T07:04:15.055335+0000","last_fullsized":"2024-04-22T07:04:15.055335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782104+0000","last_change":"2024-04-22T07:04:13.799772+0000","last_active":"2024-04-22T07:04:14.782104+0000","last_peered":"2024-04-22T07:04:14.782104+0000","last_clean":"2024-04-22T07:04:14.782104+0000","last_became_active":"2024-04-22T07:04:13.799504+0000","last_became_peered":"2024-04-22T07:04:13.799504+0000","last_unstale":"2024-04-22T07:04:14.782104+0000","last_undegraded":"2024-04-22T07:04:14.782104+0000","last_fullsized":"2024-04-22T07:04:14.782104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056133+0000","last_change":"2024-04-22T07:04:13.804604+0000","last_active":"2024-04-22T07:04:15.056133+0000","last_peered":"2024-04-22T07:04:15.056133+0000","last_clean":"2024-04-22T07:04:15.056133+0000","last_became_active":"2024-04-22T07:04:13.803594+0000","last_became_peered":"2024-04-22T07:04:13.803594+0000","last_unstale":"2024-04-22T07:04:15.056133+0000","last_undegraded":"2024-04-22T07:04:15.056133+0000","last_fullsized":"2024-04-22T07:04:15.056133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055262+0000","last_change":"2024-04-22T07:04:13.795947+0000","last_active":"2024-04-22T07:04:15.055262+0000","last_peered":"2024-04-22T07:04:15.055262+0000","last_clean":"2024-04-22T07:04:15.055262+0000","last_became_active":"2024-04-22T07:04:13.795223+0000","last_became_peered":"2024-04-22T07:04:13.795223+0000","last_unstale":"2024-04-22T07:04:15.055262+0000","last_undegraded":"2024-04-22T07:04:15.055262+0000","last_fullsized":"2024-04-22T07:04:15.055262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056195+0000","last_change":"2024-04-22T07:04:13.804225+0000","last_active":"2024-04-22T07:04:15.056195+0000","last_peered":"2024-04-22T07:04:15.056195+0000","last_clean":"2024-04-22T07:04:15.056195+0000","last_became_active":"2024-04-22T07:04:13.803920+0000","last_became_peered":"2024-04-22T07:04:13.803920+0000","last_unstale":"2024-04-22T07:04:15.056195+0000","last_undegraded":"2024-04-22T07:04:15.056195+0000","last_fullsized":"2024-04-22T07:04:15.056195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783228+0000","last_change":"2024-04-22T07:04:13.796357+0000","last_active":"2024-04-22T07:04:14.783228+0000","last_peered":"2024-04-22T07:04:14.783228+0000","last_clean":"2024-04-22T07:04:14.783228+0000","last_became_active":"2024-04-22T07:04:13.795784+0000","last_became_peered":"2024-04-22T07:04:13.795784+0000","last_unstale":"2024-04-22T07:04:14.783228+0000","last_undegraded":"2024-04-22T07:04:14.783228+0000","last_fullsized":"2024-04-22T07:04:14.783228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054855+0000","last_change":"2024-04-22T07:04:13.784337+0000","last_active":"2024-04-22T07:04:15.054855+0000","last_peered":"2024-04-22T07:04:15.054855+0000","last_clean":"2024-04-22T07:04:15.054855+0000","last_became_active":"2024-04-22T07:04:13.783914+0000","last_became_peered":"2024-04-22T07:04:13.783914+0000","last_unstale":"2024-04-22T07:04:15.054855+0000","last_undegraded":"2024-04-22T07:04:15.054855+0000","last_fullsized":"2024-04-22T07:04:15.054855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:43:41.211395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781515+0000","last_change":"2024-04-22T07:04:13.806969+0000","last_active":"2024-04-22T07:04:14.781515+0000","last_peered":"2024-04-22T07:04:14.781515+0000","last_clean":"2024-04-22T07:04:14.781515+0000","last_became_active":"2024-04-22T07:04:13.806715+0000","last_became_peered":"2024-04-22T07:04:13.806715+0000","last_unstale":"2024-04-22T07:04:14.781515+0000","last_undegraded":"2024-04-22T07:04:14.781515+0000","last_fullsized":"2024-04-22T07:04:14.781515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054895+0000","last_change":"2024-04-22T07:04:13.794896+0000","last_active":"2024-04-22T07:04:15.054895+0000","last_peered":"2024-04-22T07:04:15.054895+0000","last_clean":"2024-04-22T07:04:15.054895+0000","last_became_active":"2024-04-22T07:04:13.794605+0000","last_became_peered":"2024-04-22T07:04:13.794605+0000","last_unstale":"2024-04-22T07:04:15.054895+0000","last_undegraded":"2024-04-22T07:04:15.054895+0000","last_fullsized":"2024-04-22T07:04:15.054895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:02.613130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055503+0000","last_change":"2024-04-22T07:04:12.786738+0000","last_active":"2024-04-22T07:04:15.055503+0000","last_peered":"2024-04-22T07:04:15.055503+0000","last_clean":"2024-04-22T07:04:15.055503+0000","last_became_active":"2024-04-22T07:04:12.786413+0000","last_became_peered":"2024-04-22T07:04:12.786413+0000","last_unstale":"2024-04-22T07:04:15.055503+0000","last_undegraded":"2024-04-22T07:04:15.055503+0000","last_fullsized":"2024-04-22T07:04:15.055503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056196+0000","last_change":"2024-04-22T07:04:13.800525+0000","last_active":"2024-04-22T07:04:15.056196+0000","last_peered":"2024-04-22T07:04:15.056196+0000","last_clean":"2024-04-22T07:04:15.056196+0000","last_became_active":"2024-04-22T07:04:13.800254+0000","last_became_peered":"2024-04-22T07:04:13.800254+0000","last_unstale":"2024-04-22T07:04:15.056196+0000","last_undegraded":"2024-04-22T07:04:15.056196+0000","last_fullsized":"2024-04-22T07:04:15.056196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781988+0000","last_change":"2024-04-22T07:04:12.786726+0000","last_active":"2024-04-22T07:04:14.781988+0000","last_peered":"2024-04-22T07:04:14.781988+0000","last_clean":"2024-04-22T07:04:14.781988+0000","last_became_active":"2024-04-22T07:04:12.786444+0000","last_became_peered":"2024-04-22T07:04:12.786444+0000","last_unstale":"2024-04-22T07:04:14.781988+0000","last_undegraded":"2024-04-22T07:04:14.781988+0000","last_fullsized":"2024-04-22T07:04:14.781988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055394+0000","last_change":"2024-04-22T07:04:12.787357+0000","last_active":"2024-04-22T07:04:15.055394+0000","last_peered":"2024-04-22T07:04:15.055394+0000","last_clean":"2024-04-22T07:04:15.055394+0000","last_became_active":"2024-04-22T07:04:12.787066+0000","last_became_peered":"2024-04-22T07:04:12.787066+0000","last_unstale":"2024-04-22T07:04:15.055394+0000","last_undegraded":"2024-04-22T07:04:15.055394+0000","last_fullsized":"2024-04-22T07:04:15.055394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054713+0000","last_change":"2024-04-22T07:04:13.807697+0000","last_active":"2024-04-22T07:04:15.054713+0000","last_peered":"2024-04-22T07:04:15.054713+0000","last_clean":"2024-04-22T07:04:15.054713+0000","last_became_active":"2024-04-22T07:04:13.807327+0000","last_became_peered":"2024-04-22T07:04:13.807327+0000","last_unstale":"2024-04-22T07:04:15.054713+0000","last_undegraded":"2024-04-22T07:04:15.054713+0000","last_fullsized":"2024-04-22T07:04:15.054713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783313+0000","last_change":"2024-04-22T07:04:12.787783+0000","last_active":"2024-04-22T07:04:14.783313+0000","last_peered":"2024-04-22T07:04:14.783313+0000","last_clean":"2024-04-22T07:04:14.783313+0000","last_became_active":"2024-04-22T07:04:12.786765+0000","last_became_peered":"2024-04-22T07:04:12.786765+0000","last_unstale":"2024-04-22T07:04:14.783313+0000","last_undegraded":"2024-04-22T07:04:14.783313+0000","last_fullsized":"2024-04-22T07:04:14.783313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055740+0000","last_change":"2024-04-22T07:04:13.804931+0000","last_active":"2024-04-22T07:04:15.055740+0000","last_peered":"2024-04-22T07:04:15.055740+0000","last_clean":"2024-04-22T07:04:15.055740+0000","last_became_active":"2024-04-22T07:04:13.804629+0000","last_became_peered":"2024-04-22T07:04:13.804629+0000","last_unstale":"2024-04-22T07:04:15.055740+0000","last_undegraded":"2024-04-22T07:04:15.055740+0000","last_fullsized":"2024-04-22T07:04:15.055740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781808+0000","last_change":"2024-04-22T07:04:12.789279+0000","last_active":"2024-04-22T07:04:14.781808+0000","last_peered":"2024-04-22T07:04:14.781808+0000","last_clean":"2024-04-22T07:04:14.781808+0000","last_became_active":"2024-04-22T07:04:12.787705+0000","last_became_peered":"2024-04-22T07:04:12.787705+0000","last_unstale":"2024-04-22T07:04:14.781808+0000","last_undegraded":"2024-04-22T07:04:14.781808+0000","last_fullsized":"2024-04-22T07:04:14.781808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782812+0000","last_change":"2024-04-22T07:04:13.794292+0000","last_active":"2024-04-22T07:04:14.782812+0000","last_peered":"2024-04-22T07:04:14.782812+0000","last_clean":"2024-04-22T07:04:14.782812+0000","last_became_active":"2024-04-22T07:04:13.793986+0000","last_became_peered":"2024-04-22T07:04:13.793986+0000","last_unstale":"2024-04-22T07:04:14.782812+0000","last_undegraded":"2024-04-22T07:04:14.782812+0000","last_fullsized":"2024-04-22T07:04:14.782812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782990+0000","last_change":"2024-04-22T07:04:12.787064+0000","last_active":"2024-04-22T07:04:14.782990+0000","last_peered":"2024-04-22T07:04:14.782990+0000","last_clean":"2024-04-22T07:04:14.782990+0000","last_became_active":"2024-04-22T07:04:12.786430+0000","last_became_peered":"2024-04-22T07:04:12.786430+0000","last_unstale":"2024-04-22T07:04:14.782990+0000","last_undegraded":"2024-04-22T07:04:14.782990+0000","last_fullsized":"2024-04-22T07:04:14.782990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056732+0000","last_change":"2024-04-22T07:04:13.801337+0000","last_active":"2024-04-22T07:04:15.056732+0000","last_peered":"2024-04-22T07:04:15.056732+0000","last_clean":"2024-04-22T07:04:15.056732+0000","last_became_active":"2024-04-22T07:04:13.800916+0000","last_became_peered":"2024-04-22T07:04:13.800916+0000","last_unstale":"2024-04-22T07:04:15.056732+0000","last_undegraded":"2024-04-22T07:04:15.056732+0000","last_fullsized":"2024-04-22T07:04:15.056732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056192+0000","last_change":"2024-04-22T07:04:12.787741+0000","last_active":"2024-04-22T07:04:15.056192+0000","last_peered":"2024-04-22T07:04:15.056192+0000","last_clean":"2024-04-22T07:04:15.056192+0000","last_became_active":"2024-04-22T07:04:12.787405+0000","last_became_peered":"2024-04-22T07:04:12.787405+0000","last_unstale":"2024-04-22T07:04:15.056192+0000","last_undegraded":"2024-04-22T07:04:15.056192+0000","last_fullsized":"2024-04-22T07:04:15.056192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055184+0000","last_change":"2024-04-22T07:04:13.795502+0000","last_active":"2024-04-22T07:04:15.055184+0000","last_peered":"2024-04-22T07:04:15.055184+0000","last_clean":"2024-04-22T07:04:15.055184+0000","last_became_active":"2024-04-22T07:04:13.795199+0000","last_became_peered":"2024-04-22T07:04:13.795199+0000","last_unstale":"2024-04-22T07:04:15.055184+0000","last_undegraded":"2024-04-22T07:04:15.055184+0000","last_fullsized":"2024-04-22T07:04:15.055184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:07.780008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781679+0000","last_change":"2024-04-22T07:04:12.788572+0000","last_active":"2024-04-22T07:04:14.781679+0000","last_peered":"2024-04-22T07:04:14.781679+0000","last_clean":"2024-04-22T07:04:14.781679+0000","last_became_active":"2024-04-22T07:04:12.788286+0000","last_became_peered":"2024-04-22T07:04:12.788286+0000","last_unstale":"2024-04-22T07:04:14.781679+0000","last_undegraded":"2024-04-22T07:04:14.781679+0000","last_fullsized":"2024-04-22T07:04:14.781679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961160+0000","last_change":"2024-04-22T07:04:13.804435+0000","last_active":"2024-04-22T07:04:14.961160+0000","last_peered":"2024-04-22T07:04:14.961160+0000","last_clean":"2024-04-22T07:04:14.961160+0000","last_became_active":"2024-04-22T07:04:13.804170+0000","last_became_peered":"2024-04-22T07:04:13.804170+0000","last_unstale":"2024-04-22T07:04:14.961160+0000","last_undegraded":"2024-04-22T07:04:14.961160+0000","last_fullsized":"2024-04-22T07:04:14.961160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056278+0000","last_change":"2024-04-22T07:04:12.789905+0000","last_active":"2024-04-22T07:04:15.056278+0000","last_peered":"2024-04-22T07:04:15.056278+0000","last_clean":"2024-04-22T07:04:15.056278+0000","last_became_active":"2024-04-22T07:04:12.789503+0000","last_became_peered":"2024-04-22T07:04:12.789503+0000","last_unstale":"2024-04-22T07:04:15.056278+0000","last_undegraded":"2024-04-22T07:04:15.056278+0000","last_fullsized":"2024-04-22T07:04:15.056278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055741+0000","last_change":"2024-04-22T07:04:13.801490+0000","last_active":"2024-04-22T07:04:15.055741+0000","last_peered":"2024-04-22T07:04:15.055741+0000","last_clean":"2024-04-22T07:04:15.055741+0000","last_became_active":"2024-04-22T07:04:13.799356+0000","last_became_peered":"2024-04-22T07:04:13.799356+0000","last_unstale":"2024-04-22T07:04:15.055741+0000","last_undegraded":"2024-04-22T07:04:15.055741+0000","last_fullsized":"2024-04-22T07:04:15.055741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:32:04.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783198+0000","last_change":"2024-04-22T07:04:12.781089+0000","last_active":"2024-04-22T07:04:14.783198+0000","last_peered":"2024-04-22T07:04:14.783198+0000","last_clean":"2024-04-22T07:04:14.783198+0000","last_became_active":"2024-04-22T07:04:12.780827+0000","last_became_peered":"2024-04-22T07:04:12.780827+0000","last_unstale":"2024-04-22T07:04:14.783198+0000","last_undegraded":"2024-04-22T07:04:14.783198+0000","last_fullsized":"2024-04-22T07:04:14.783198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961202+0000","last_change":"2024-04-22T07:04:13.801375+0000","last_active":"2024-04-22T07:04:14.961202+0000","last_peered":"2024-04-22T07:04:14.961202+0000","last_clean":"2024-04-22T07:04:14.961202+0000","last_became_active":"2024-04-22T07:04:13.799488+0000","last_became_peered":"2024-04-22T07:04:13.799488+0000","last_unstale":"2024-04-22T07:04:14.961202+0000","last_undegraded":"2024-04-22T07:04:14.961202+0000","last_fullsized":"2024-04-22T07:04:14.961202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781576+0000","last_change":"2024-04-22T07:04:12.790758+0000","last_active":"2024-04-22T07:04:14.781576+0000","last_peered":"2024-04-22T07:04:14.781576+0000","last_clean":"2024-04-22T07:04:14.781576+0000","last_became_active":"2024-04-22T07:04:12.790444+0000","last_became_peered":"2024-04-22T07:04:12.790444+0000","last_unstale":"2024-04-22T07:04:14.781576+0000","last_undegraded":"2024-04-22T07:04:14.781576+0000","last_fullsized":"2024-04-22T07:04:14.781576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782297+0000","last_change":"2024-04-22T07:04:13.805415+0000","last_active":"2024-04-22T07:04:14.782297+0000","last_peered":"2024-04-22T07:04:14.782297+0000","last_clean":"2024-04-22T07:04:14.782297+0000","last_became_active":"2024-04-22T07:04:13.805115+0000","last_became_peered":"2024-04-22T07:04:13.805115+0000","last_unstale":"2024-04-22T07:04:14.782297+0000","last_undegraded":"2024-04-22T07:04:14.782297+0000","last_fullsized":"2024-04-22T07:04:14.782297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054450+0000","last_change":"2024-04-22T07:04:12.789877+0000","last_active":"2024-04-22T07:04:15.054450+0000","last_peered":"2024-04-22T07:04:15.054450+0000","last_clean":"2024-04-22T07:04:15.054450+0000","last_became_active":"2024-04-22T07:04:12.789546+0000","last_became_peered":"2024-04-22T07:04:12.789546+0000","last_unstale":"2024-04-22T07:04:15.054450+0000","last_undegraded":"2024-04-22T07:04:15.054450+0000","last_fullsized":"2024-04-22T07:04:15.054450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:04.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961082+0000","last_change":"2024-04-22T07:04:13.801803+0000","last_active":"2024-04-22T07:04:14.961082+0000","last_peered":"2024-04-22T07:04:14.961082+0000","last_clean":"2024-04-22T07:04:14.961082+0000","last_became_active":"2024-04-22T07:04:13.801362+0000","last_became_peered":"2024-04-22T07:04:13.801362+0000","last_unstale":"2024-04-22T07:04:14.961082+0000","last_undegraded":"2024-04-22T07:04:14.961082+0000","last_fullsized":"2024-04-22T07:04:14.961082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055197+0000","last_change":"2024-04-22T07:04:12.787917+0000","last_active":"2024-04-22T07:04:15.055197+0000","last_peered":"2024-04-22T07:04:15.055197+0000","last_clean":"2024-04-22T07:04:15.055197+0000","last_became_active":"2024-04-22T07:04:12.787636+0000","last_became_peered":"2024-04-22T07:04:12.787636+0000","last_unstale":"2024-04-22T07:04:15.055197+0000","last_undegraded":"2024-04-22T07:04:15.055197+0000","last_fullsized":"2024-04-22T07:04:15.055197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961456+0000","last_change":"2024-04-22T07:04:13.802786+0000","last_active":"2024-04-22T07:04:14.961456+0000","last_peered":"2024-04-22T07:04:14.961456+0000","last_clean":"2024-04-22T07:04:14.961456+0000","last_became_active":"2024-04-22T07:04:13.801299+0000","last_became_peered":"2024-04-22T07:04:13.801299+0000","last_unstale":"2024-04-22T07:04:14.961456+0000","last_undegraded":"2024-04-22T07:04:14.961456+0000","last_fullsized":"2024-04-22T07:04:14.961456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:28.362520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054920+0000","last_change":"2024-04-22T07:04:12.788593+0000","last_active":"2024-04-22T07:04:15.054920+0000","last_peered":"2024-04-22T07:04:15.054920+0000","last_clean":"2024-04-22T07:04:15.054920+0000","last_became_active":"2024-04-22T07:04:12.788314+0000","last_became_peered":"2024-04-22T07:04:12.788314+0000","last_unstale":"2024-04-22T07:04:15.054920+0000","last_undegraded":"2024-04-22T07:04:15.054920+0000","last_fullsized":"2024-04-22T07:04:15.054920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:11:27.081266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961608+0000","last_change":"2024-04-22T07:04:13.800245+0000","last_active":"2024-04-22T07:04:14.961608+0000","last_peered":"2024-04-22T07:04:14.961608+0000","last_clean":"2024-04-22T07:04:14.961608+0000","last_became_active":"2024-04-22T07:04:13.799970+0000","last_became_peered":"2024-04-22T07:04:13.799970+0000","last_unstale":"2024-04-22T07:04:14.961608+0000","last_undegraded":"2024-04-22T07:04:14.961608+0000","last_fullsized":"2024-04-22T07:04:14.961608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:13.558356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056239+0000","last_change":"2024-04-22T07:04:12.790003+0000","last_active":"2024-04-22T07:04:15.056239+0000","last_peered":"2024-04-22T07:04:15.056239+0000","last_clean":"2024-04-22T07:04:15.056239+0000","last_became_active":"2024-04-22T07:04:12.789708+0000","last_became_peered":"2024-04-22T07:04:12.789708+0000","last_unstale":"2024-04-22T07:04:15.056239+0000","last_undegraded":"2024-04-22T07:04:15.056239+0000","last_fullsized":"2024-04-22T07:04:15.056239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783345+0000","last_change":"2024-04-22T07:04:13.795082+0000","last_active":"2024-04-22T07:04:14.783345+0000","last_peered":"2024-04-22T07:04:14.783345+0000","last_clean":"2024-04-22T07:04:14.783345+0000","last_became_active":"2024-04-22T07:04:13.794793+0000","last_became_peered":"2024-04-22T07:04:13.794793+0000","last_unstale":"2024-04-22T07:04:14.783345+0000","last_undegraded":"2024-04-22T07:04:14.783345+0000","last_fullsized":"2024-04-22T07:04:14.783345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054608+0000","last_change":"2024-04-22T07:04:12.789252+0000","last_active":"2024-04-22T07:04:15.054608+0000","last_peered":"2024-04-22T07:04:15.054608+0000","last_clean":"2024-04-22T07:04:15.054608+0000","last_became_active":"2024-04-22T07:04:12.788946+0000","last_became_peered":"2024-04-22T07:04:12.788946+0000","last_unstale":"2024-04-22T07:04:15.054608+0000","last_undegraded":"2024-04-22T07:04:15.054608+0000","last_fullsized":"2024-04-22T07:04:15.054608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:36:13.428703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961272+0000","last_change":"2024-04-22T07:04:13.801250+0000","last_active":"2024-04-22T07:04:14.961272+0000","last_peered":"2024-04-22T07:04:14.961272+0000","last_clean":"2024-04-22T07:04:14.961272+0000","last_became_active":"2024-04-22T07:04:13.799143+0000","last_became_peered":"2024-04-22T07:04:13.799143+0000","last_unstale":"2024-04-22T07:04:14.961272+0000","last_undegraded":"2024-04-22T07:04:14.961272+0000","last_fullsized":"2024-04-22T07:04:14.961272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960952+0000","last_change":"2024-04-22T07:04:03.688462+0000","last_active":"2024-04-22T07:04:14.960952+0000","last_peered":"2024-04-22T07:04:14.960952+0000","last_clean":"2024-04-22T07:04:14.960952+0000","last_became_active":"2024-04-22T07:04:03.688191+0000","last_became_peered":"2024-04-22T07:04:03.688191+0000","last_unstale":"2024-04-22T07:04:14.960952+0000","last_undegraded":"2024-04-22T07:04:14.960952+0000","last_fullsized":"2024-04-22T07:04:14.960952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.464330+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.053999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782468+0000","last_change":"2024-04-22T07:04:12.783751+0000","last_active":"2024-04-22T07:04:14.782468+0000","last_peered":"2024-04-22T07:04:14.782468+0000","last_clean":"2024-04-22T07:04:14.782468+0000","last_became_active":"2024-04-22T07:04:12.783460+0000","last_became_peered":"2024-04-22T07:04:12.783460+0000","last_unstale":"2024-04-22T07:04:14.782468+0000","last_undegraded":"2024-04-22T07:04:14.782468+0000","last_fullsized":"2024-04-22T07:04:14.782468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054881+0000","last_change":"2024-04-22T07:04:13.791961+0000","last_active":"2024-04-22T07:04:15.054881+0000","last_peered":"2024-04-22T07:04:15.054881+0000","last_clean":"2024-04-22T07:04:15.054881+0000","last_became_active":"2024-04-22T07:04:13.790591+0000","last_became_peered":"2024-04-22T07:04:13.790591+0000","last_unstale":"2024-04-22T07:04:15.054881+0000","last_undegraded":"2024-04-22T07:04:15.054881+0000","last_fullsized":"2024-04-22T07:04:15.054881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960999+0000","last_change":"2024-04-22T07:04:12.787268+0000","last_active":"2024-04-22T07:04:14.960999+0000","last_peered":"2024-04-22T07:04:14.960999+0000","last_clean":"2024-04-22T07:04:14.960999+0000","last_became_active":"2024-04-22T07:04:12.786991+0000","last_became_peered":"2024-04-22T07:04:12.786991+0000","last_unstale":"2024-04-22T07:04:14.960999+0000","last_undegraded":"2024-04-22T07:04:14.960999+0000","last_fullsized":"2024-04-22T07:04:14.960999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054726+0000","last_change":"2024-04-22T07:04:13.800243+0000","last_active":"2024-04-22T07:04:15.054726+0000","last_peered":"2024-04-22T07:04:15.054726+0000","last_clean":"2024-04-22T07:04:15.054726+0000","last_became_active":"2024-04-22T07:04:13.799917+0000","last_became_peered":"2024-04-22T07:04:13.799917+0000","last_unstale":"2024-04-22T07:04:15.054726+0000","last_undegraded":"2024-04-22T07:04:15.054726+0000","last_fullsized":"2024-04-22T07:04:15.054726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:54.958916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054937+0000","last_change":"2024-04-22T07:04:12.787893+0000","last_active":"2024-04-22T07:04:15.054937+0000","last_peered":"2024-04-22T07:04:15.054937+0000","last_clean":"2024-04-22T07:04:15.054937+0000","last_became_active":"2024-04-22T07:04:12.787370+0000","last_became_peered":"2024-04-22T07:04:12.787370+0000","last_unstale":"2024-04-22T07:04:15.054937+0000","last_undegraded":"2024-04-22T07:04:15.054937+0000","last_fullsized":"2024-04-22T07:04:15.054937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:52:03.220670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055952+0000","last_change":"2024-04-22T07:04:13.795454+0000","last_active":"2024-04-22T07:04:15.055952+0000","last_peered":"2024-04-22T07:04:15.055952+0000","last_clean":"2024-04-22T07:04:15.055952+0000","last_became_active":"2024-04-22T07:04:13.795138+0000","last_became_peered":"2024-04-22T07:04:13.795138+0000","last_unstale":"2024-04-22T07:04:15.055952+0000","last_undegraded":"2024-04-22T07:04:15.055952+0000","last_fullsized":"2024-04-22T07:04:15.055952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783273+0000","last_change":"2024-04-22T07:04:12.786891+0000","last_active":"2024-04-22T07:04:14.783273+0000","last_peered":"2024-04-22T07:04:14.783273+0000","last_clean":"2024-04-22T07:04:14.783273+0000","last_became_active":"2024-04-22T07:04:12.786557+0000","last_became_peered":"2024-04-22T07:04:12.786557+0000","last_unstale":"2024-04-22T07:04:14.783273+0000","last_undegraded":"2024-04-22T07:04:14.783273+0000","last_fullsized":"2024-04-22T07:04:14.783273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055054+0000","last_change":"2024-04-22T07:04:13.798678+0000","last_active":"2024-04-22T07:04:15.055054+0000","last_peered":"2024-04-22T07:04:15.055054+0000","last_clean":"2024-04-22T07:04:15.055054+0000","last_became_active":"2024-04-22T07:04:13.798369+0000","last_became_peered":"2024-04-22T07:04:13.798369+0000","last_unstale":"2024-04-22T07:04:15.055054+0000","last_undegraded":"2024-04-22T07:04:15.055054+0000","last_fullsized":"2024-04-22T07:04:15.055054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:37.919431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055841+0000","last_change":"2024-04-22T07:04:12.791897+0000","last_active":"2024-04-22T07:04:15.055841+0000","last_peered":"2024-04-22T07:04:15.055841+0000","last_clean":"2024-04-22T07:04:15.055841+0000","last_became_active":"2024-04-22T07:04:12.791580+0000","last_became_peered":"2024-04-22T07:04:12.791580+0000","last_unstale":"2024-04-22T07:04:15.055841+0000","last_undegraded":"2024-04-22T07:04:15.055841+0000","last_fullsized":"2024-04-22T07:04:15.055841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782942+0000","last_change":"2024-04-22T07:04:13.803516+0000","last_active":"2024-04-22T07:04:14.782942+0000","last_peered":"2024-04-22T07:04:14.782942+0000","last_clean":"2024-04-22T07:04:14.782942+0000","last_became_active":"2024-04-22T07:04:13.803230+0000","last_became_peered":"2024-04-22T07:04:13.803230+0000","last_unstale":"2024-04-22T07:04:14.782942+0000","last_undegraded":"2024-04-22T07:04:14.782942+0000","last_fullsized":"2024-04-22T07:04:14.782942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961432+0000","last_change":"2024-04-22T07:04:12.789499+0000","last_active":"2024-04-22T07:04:14.961432+0000","last_peered":"2024-04-22T07:04:14.961432+0000","last_clean":"2024-04-22T07:04:14.961432+0000","last_became_active":"2024-04-22T07:04:12.789229+0000","last_became_peered":"2024-04-22T07:04:12.789229+0000","last_unstale":"2024-04-22T07:04:14.961432+0000","last_undegraded":"2024-04-22T07:04:14.961432+0000","last_fullsized":"2024-04-22T07:04:14.961432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:53.955113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055856+0000","last_change":"2024-04-22T07:04:13.797969+0000","last_active":"2024-04-22T07:04:15.055856+0000","last_peered":"2024-04-22T07:04:15.055856+0000","last_clean":"2024-04-22T07:04:15.055856+0000","last_became_active":"2024-04-22T07:04:13.797662+0000","last_became_peered":"2024-04-22T07:04:13.797662+0000","last_unstale":"2024-04-22T07:04:15.055856+0000","last_undegraded":"2024-04-22T07:04:15.055856+0000","last_fullsized":"2024-04-22T07:04:15.055856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056169+0000","last_change":"2024-04-22T07:04:12.791787+0000","last_active":"2024-04-22T07:04:15.056169+0000","last_peered":"2024-04-22T07:04:15.056169+0000","last_clean":"2024-04-22T07:04:15.056169+0000","last_became_active":"2024-04-22T07:04:12.791460+0000","last_became_peered":"2024-04-22T07:04:12.791460+0000","last_unstale":"2024-04-22T07:04:15.056169+0000","last_undegraded":"2024-04-22T07:04:15.056169+0000","last_fullsized":"2024-04-22T07:04:15.056169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055451+0000","last_change":"2024-04-22T07:04:13.797755+0000","last_active":"2024-04-22T07:04:15.055451+0000","last_peered":"2024-04-22T07:04:15.055451+0000","last_clean":"2024-04-22T07:04:15.055451+0000","last_became_active":"2024-04-22T07:04:13.797400+0000","last_became_peered":"2024-04-22T07:04:13.797400+0000","last_unstale":"2024-04-22T07:04:15.055451+0000","last_undegraded":"2024-04-22T07:04:15.055451+0000","last_fullsized":"2024-04-22T07:04:15.055451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781905+0000","last_change":"2024-04-22T07:04:12.790458+0000","last_active":"2024-04-22T07:04:14.781905+0000","last_peered":"2024-04-22T07:04:14.781905+0000","last_clean":"2024-04-22T07:04:14.781905+0000","last_became_active":"2024-04-22T07:04:12.790209+0000","last_became_peered":"2024-04-22T07:04:12.790209+0000","last_unstale":"2024-04-22T07:04:14.781905+0000","last_undegraded":"2024-04-22T07:04:14.781905+0000","last_fullsized":"2024-04-22T07:04:14.781905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056058+0000","last_change":"2024-04-22T07:04:13.796176+0000","last_active":"2024-04-22T07:04:15.056058+0000","last_peered":"2024-04-22T07:04:15.056058+0000","last_clean":"2024-04-22T07:04:15.056058+0000","last_became_active":"2024-04-22T07:04:13.795325+0000","last_became_peered":"2024-04-22T07:04:13.795325+0000","last_unstale":"2024-04-22T07:04:15.056058+0000","last_undegraded":"2024-04-22T07:04:15.056058+0000","last_fullsized":"2024-04-22T07:04:15.056058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960859+0000","last_change":"2024-04-22T07:04:12.791157+0000","last_active":"2024-04-22T07:04:14.960859+0000","last_peered":"2024-04-22T07:04:14.960859+0000","last_clean":"2024-04-22T07:04:14.960859+0000","last_became_active":"2024-04-22T07:04:12.790879+0000","last_became_peered":"2024-04-22T07:04:12.790879+0000","last_unstale":"2024-04-22T07:04:14.960859+0000","last_undegraded":"2024-04-22T07:04:14.960859+0000","last_fullsized":"2024-04-22T07:04:14.960859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054928+0000","last_change":"2024-04-22T07:04:13.796587+0000","last_active":"2024-04-22T07:04:15.054928+0000","last_peered":"2024-04-22T07:04:15.054928+0000","last_clean":"2024-04-22T07:04:15.054928+0000","last_became_active":"2024-04-22T07:04:13.796305+0000","last_became_peered":"2024-04-22T07:04:13.796305+0000","last_unstale":"2024-04-22T07:04:15.054928+0000","last_undegraded":"2024-04-22T07:04:15.054928+0000","last_fullsized":"2024-04-22T07:04:15.054928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:29.793237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055857+0000","last_change":"2024-04-22T07:04:12.784359+0000","last_active":"2024-04-22T07:04:15.055857+0000","last_peered":"2024-04-22T07:04:15.055857+0000","last_clean":"2024-04-22T07:04:15.055857+0000","last_became_active":"2024-04-22T07:04:12.784076+0000","last_became_peered":"2024-04-22T07:04:12.784076+0000","last_unstale":"2024-04-22T07:04:15.055857+0000","last_undegraded":"2024-04-22T07:04:15.055857+0000","last_fullsized":"2024-04-22T07:04:15.055857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054875+0000","last_change":"2024-04-22T07:04:13.802658+0000","last_active":"2024-04-22T07:04:15.054875+0000","last_peered":"2024-04-22T07:04:15.054875+0000","last_clean":"2024-04-22T07:04:15.054875+0000","last_became_active":"2024-04-22T07:04:13.802059+0000","last_became_peered":"2024-04-22T07:04:13.802059+0000","last_unstale":"2024-04-22T07:04:15.054875+0000","last_undegraded":"2024-04-22T07:04:15.054875+0000","last_fullsized":"2024-04-22T07:04:15.054875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055829+0000","last_change":"2024-04-22T07:04:12.788090+0000","last_active":"2024-04-22T07:04:15.055829+0000","last_peered":"2024-04-22T07:04:15.055829+0000","last_clean":"2024-04-22T07:04:15.055829+0000","last_became_active":"2024-04-22T07:04:12.787818+0000","last_became_peered":"2024-04-22T07:04:12.787818+0000","last_unstale":"2024-04-22T07:04:15.055829+0000","last_undegraded":"2024-04-22T07:04:15.055829+0000","last_fullsized":"2024-04-22T07:04:15.055829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055442+0000","last_change":"2024-04-22T07:04:13.802013+0000","last_active":"2024-04-22T07:04:15.055442+0000","last_peered":"2024-04-22T07:04:15.055442+0000","last_clean":"2024-04-22T07:04:15.055442+0000","last_became_active":"2024-04-22T07:04:13.801404+0000","last_became_peered":"2024-04-22T07:04:13.801404+0000","last_unstale":"2024-04-22T07:04:15.055442+0000","last_undegraded":"2024-04-22T07:04:15.055442+0000","last_fullsized":"2024-04-22T07:04:15.055442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961382+0000","last_change":"2024-04-22T07:04:12.790981+0000","last_active":"2024-04-22T07:04:14.961382+0000","last_peered":"2024-04-22T07:04:14.961382+0000","last_clean":"2024-04-22T07:04:14.961382+0000","last_became_active":"2024-04-22T07:04:12.790592+0000","last_became_peered":"2024-04-22T07:04:12.790592+0000","last_unstale":"2024-04-22T07:04:14.961382+0000","last_undegraded":"2024-04-22T07:04:14.961382+0000","last_fullsized":"2024-04-22T07:04:14.961382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:53.115405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781878+0000","last_change":"2024-04-22T07:04:13.800619+0000","last_active":"2024-04-22T07:04:14.781878+0000","last_peered":"2024-04-22T07:04:14.781878+0000","last_clean":"2024-04-22T07:04:14.781878+0000","last_became_active":"2024-04-22T07:04:13.800279+0000","last_became_peered":"2024-04-22T07:04:13.800279+0000","last_unstale":"2024-04-22T07:04:14.781878+0000","last_undegraded":"2024-04-22T07:04:14.781878+0000","last_fullsized":"2024-04-22T07:04:14.781878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782949+0000","last_change":"2024-04-22T07:04:13.789060+0000","last_active":"2024-04-22T07:04:14.782949+0000","last_peered":"2024-04-22T07:04:14.782949+0000","last_clean":"2024-04-22T07:04:14.782949+0000","last_became_active":"2024-04-22T07:04:13.788787+0000","last_became_peered":"2024-04-22T07:04:13.788787+0000","last_unstale":"2024-04-22T07:04:14.782949+0000","last_undegraded":"2024-04-22T07:04:14.782949+0000","last_fullsized":"2024-04-22T07:04:14.782949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961028+0000","last_change":"2024-04-22T07:04:12.785077+0000","last_active":"2024-04-22T07:04:14.961028+0000","last_peered":"2024-04-22T07:04:14.961028+0000","last_clean":"2024-04-22T07:04:14.961028+0000","last_became_active":"2024-04-22T07:04:12.784804+0000","last_became_peered":"2024-04-22T07:04:12.784804+0000","last_unstale":"2024-04-22T07:04:14.961028+0000","last_undegraded":"2024-04-22T07:04:14.961028+0000","last_fullsized":"2024-04-22T07:04:14.961028+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783259+0000","last_change":"2024-04-22T07:04:13.789866+0000","last_active":"2024-04-22T07:04:14.783259+0000","last_peered":"2024-04-22T07:04:14.783259+0000","last_clean":"2024-04-22T07:04:14.783259+0000","last_became_active":"2024-04-22T07:04:13.789539+0000","last_became_peered":"2024-04-22T07:04:13.789539+0000","last_unstale":"2024-04-22T07:04:14.783259+0000","last_undegraded":"2024-04-22T07:04:14.783259+0000","last_fullsized":"2024-04-22T07:04:14.783259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961290+0000","last_change":"2024-04-22T07:04:12.786162+0000","last_active":"2024-04-22T07:04:14.961290+0000","last_peered":"2024-04-22T07:04:14.961290+0000","last_clean":"2024-04-22T07:04:14.961290+0000","last_became_active":"2024-04-22T07:04:12.785310+0000","last_became_peered":"2024-04-22T07:04:12.785310+0000","last_unstale":"2024-04-22T07:04:14.961290+0000","last_undegraded":"2024-04-22T07:04:14.961290+0000","last_fullsized":"2024-04-22T07:04:14.961290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:06.776205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056634+0000","last_change":"2024-04-22T07:04:13.796267+0000","last_active":"2024-04-22T07:04:15.056634+0000","last_peered":"2024-04-22T07:04:15.056634+0000","last_clean":"2024-04-22T07:04:15.056634+0000","last_became_active":"2024-04-22T07:04:13.795220+0000","last_became_peered":"2024-04-22T07:04:13.795220+0000","last_unstale":"2024-04-22T07:04:15.056634+0000","last_undegraded":"2024-04-22T07:04:15.056634+0000","last_fullsized":"2024-04-22T07:04:15.056634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055546+0000","last_change":"2024-04-22T07:04:12.785962+0000","last_active":"2024-04-22T07:04:15.055546+0000","last_peered":"2024-04-22T07:04:15.055546+0000","last_clean":"2024-04-22T07:04:15.055546+0000","last_became_active":"2024-04-22T07:04:12.785608+0000","last_became_peered":"2024-04-22T07:04:12.785608+0000","last_unstale":"2024-04-22T07:04:15.055546+0000","last_undegraded":"2024-04-22T07:04:15.055546+0000","last_fullsized":"2024-04-22T07:04:15.055546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055613+0000","last_change":"2024-04-22T07:04:13.801662+0000","last_active":"2024-04-22T07:04:15.055613+0000","last_peered":"2024-04-22T07:04:15.055613+0000","last_clean":"2024-04-22T07:04:15.055613+0000","last_became_active":"2024-04-22T07:04:13.798282+0000","last_became_peered":"2024-04-22T07:04:13.798282+0000","last_unstale":"2024-04-22T07:04:15.055613+0000","last_undegraded":"2024-04-22T07:04:15.055613+0000","last_fullsized":"2024-04-22T07:04:15.055613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056742+0000","last_change":"2024-04-22T07:04:12.791445+0000","last_active":"2024-04-22T07:04:15.056742+0000","last_peered":"2024-04-22T07:04:15.056742+0000","last_clean":"2024-04-22T07:04:15.056742+0000","last_became_active":"2024-04-22T07:04:12.791150+0000","last_became_peered":"2024-04-22T07:04:12.791150+0000","last_unstale":"2024-04-22T07:04:15.056742+0000","last_undegraded":"2024-04-22T07:04:15.056742+0000","last_fullsized":"2024-04-22T07:04:15.056742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":827392,"data_stored":671603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":221184,"data_stored":74935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":671603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:19.807 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.944+0000 7fcc25832640 1 Processor -- start 2024-04-22T07:04:19.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.944+0000 7fcc25832640 1 -- start start 2024-04-22T07:04:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.945+0000 7fcc25832640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc20154f70 0x7fcc200d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.945+0000 7fcc25832640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc20155350 0x7fcc200d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.945+0000 7fcc25832640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc2015b180 con 0x7fcc20072d10 2024-04-22T07:04:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.945+0000 7fcc25832640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fcc2015b460 con 0x7fcc20155350 2024-04-22T07:04:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.945+0000 7fcc25832640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2015b740 con 0x7fcc20154f70 2024-04-22T07:04:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.945+0000 7fcc1f7fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fcc20072d10 0x7fcc200b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38014/0 (socket says 172.21.15.44:38014) 2024-04-22T07:04:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.945+0000 7fcc1f7fe640 1 -- 172.21.15.44:0/1718805321 learned_addr learned my addr 172.21.15.44:0/1718805321 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:19.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.945+0000 7fcc1effd640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc20154f70 0x7fcc200d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.945+0000 7fcc1e7fc640 1 --2- 172.21.15.44:0/1718805321 >> v2:172.21.15.70:3300/0 conn(0x7fcc20155350 0x7fcc200d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4124072141 0 0) 0x7fcc2015b180 con 0x7fcc20072d10 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc0c003520 con 0x7fcc20072d10 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1040142078 0 0) 0x7fcc0c003520 con 0x7fcc20072d10 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc20154f70 msgr2=0x7fcc200d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc20154f70 0x7fcc200d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 >> v2:172.21.15.70:3300/0 conn(0x7fcc20155350 msgr2=0x7fcc200d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 --2- 172.21.15.44:0/1718805321 >> v2:172.21.15.70:3300/0 conn(0x7fcc20155350 0x7fcc200d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2015ba20 con 0x7fcc20072d10 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3718881788 0 0) 0x7fcc100019f0 con 0x7fcc20072d10 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 >> v1:172.21.15.44:6789/0 conn(0x7fcc20072d10 legacy=0x7fcc200b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc20154f70 0x7fcc0c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c003f30 0x7fcc0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0c004d40 0x7fcc0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2015b460 con 0x7fcc0c004d40 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2015b740 con 0x7fcc0c003f30 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcc0c00b9c0 con 0x7fcc20154f70 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1effd640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc20154f70 0x7fcc0c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1f7fe640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0c004d40 0x7fcc0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1e7fc640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c003f30 0x7fcc0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1f7fe640 1 -- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc20154f70 msgr2=0x7fcc0c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1f7fe640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc20154f70 0x7fcc0c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1f7fe640 1 -- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c003f30 msgr2=0x7fcc0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1f7fe640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c003f30 0x7fcc0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1f7fe640 1 -- 172.21.15.44:0/1718805321 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c003520 con 0x7fcc0c004d40 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.946+0000 7fcc1effd640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc20154f70 0x7fcc0c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.947+0000 7fcc1f7fe640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0c004d40 0x7fcc0c0051c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000a880 tx=0x7fcc0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d8fb5aef5b93b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:19.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.947+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000a4a0 con 0x7fcc0c004d40 2024-04-22T07:04:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.947+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc10034650 con 0x7fcc0c004d40 2024-04-22T07:04:19.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.947+0000 7fcc1dffb640 1 -- 172.21.15.44:0/1718805321 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10034a40 con 0x7fcc0c004d40 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.948+0000 7fcc25832640 1 -- 172.21.15.44:0/1718805321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0c004d40 msgr2=0x7fcc0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.948+0000 7fcc25832640 1 --2- 172.21.15.44:0/1718805321 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc0c004d40 0x7fcc0c0051c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000a880 tx=0x7fcc0c005830 comp rx=0 tx=0).stop 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.948+0000 7fcc1f7fe640 1 -- 172.21.15.44:0/1718805321 reap_dead start 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.948+0000 7fcc25832640 1 -- 172.21.15.44:0/1718805321 shutdown_connections 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.948+0000 7fcc25832640 1 -- 172.21.15.44:0/1718805321 >> 172.21.15.44:0/1718805321 conn(0x7fcc200735c0 msgr2=0x7fcc200739d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.948+0000 7fcc25832640 1 -- 172.21.15.44:0/1718805321 shutdown_connections 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.948+0000 7fcc25832640 1 -- 172.21.15.44:0/1718805321 wait complete. 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.949+0000 7fcc25832640 1 Processor -- start 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.949+0000 7fcc25832640 1 -- start start 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc25832640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c004d40 0x7fcc200c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc25832640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc201e4250 0x7fcc201e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc25832640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc2015d8c0 0x7fcc201e5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc25832640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2015add0 con 0x7fcc201e4250 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc25832640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcc200caf30 con 0x7fcc0c004d40 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc25832640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fcc201e5810 con 0x7fcc2015d8c0 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc201e4250 0x7fcc201e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1e7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc201e4250 0x7fcc201e78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37856/0 (socket says 172.21.15.44:37856) 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1e7fc640 1 -- 172.21.15.44:0/3835981325 learned_addr learned my addr 172.21.15.44:0/3835981325 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1f7fe640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc2015d8c0 0x7fcc201e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1effd640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c004d40 0x7fcc200c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1e7fc640 1 -- 172.21.15.44:0/3835981325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc2015d8c0 msgr2=0x7fcc201e5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1e7fc640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc2015d8c0 0x7fcc201e5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1e7fc640 1 -- 172.21.15.44:0/3835981325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c004d40 msgr2=0x7fcc200c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1e7fc640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c004d40 0x7fcc200c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:19.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1e7fc640 1 -- 172.21.15.44:0/3835981325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc201e7f40 con 0x7fcc201e4250 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1f7fe640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc2015d8c0 0x7fcc201e5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.950+0000 7fcc1effd640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c004d40 0x7fcc200c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.951+0000 7fcc1e7fc640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc201e4250 0x7fcc201e78b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcc08011960 tx=0x7fcc08011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.951+0000 7fcc17fff640 1 -- 172.21.15.44:0/3835981325 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0800faf0 con 0x7fcc201e4250 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.951+0000 7fcc17fff640 1 -- 172.21.15.44:0/3835981325 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc0800fc90 con 0x7fcc201e4250 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.951+0000 7fcc25832640 1 -- 172.21.15.44:0/3835981325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc201f4570 con 0x7fcc201e4250 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.951+0000 7fcc17fff640 1 -- 172.21.15.44:0/3835981325 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08014440 con 0x7fcc201e4250 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.951+0000 7fcc25832640 1 -- 172.21.15.44:0/3835981325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc201f4b30 con 0x7fcc201e4250 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.952+0000 7fcc25832640 1 -- 172.21.15.44:0/3835981325 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc200b35d0 con 0x7fcc201e4250 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.952+0000 7fcc17fff640 1 -- 172.21.15.44:0/3835981325 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc08012040 con 0x7fcc201e4250 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.952+0000 7fcc17fff640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc0006c920 0x7fcc0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.953+0000 7fcc17fff640 1 -- 172.21.15.44:0/3835981325 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcc0808c290 con 0x7fcc201e4250 2024-04-22T07:04:19.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.953+0000 7fcc1effd640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc0006c920 0x7fcc0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:19.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.953+0000 7fcc1effd640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc0006c920 0x7fcc0006ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009d60 tx=0x7fcc18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:19.955 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:19.957+0000 7fcc17fff640 1 -- 172.21.15.44:0/3835981325 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc200b35d0 con 0x7fcc201e4250 2024-04-22T07:04:20.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.138+0000 7fcc25832640 1 -- 172.21.15.44:0/3835981325 --> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc200c63f0 con 0x7fcc0006c920 2024-04-22T07:04:20.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.144+0000 7fcc17fff640 1 -- 172.21.15.44:0/3835981325 <== mgr.4101 v2:172.21.15.44:6832/1889438723 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251849 (secure 0 0 0) 0x7fcc200c63f0 con 0x7fcc0006c920 2024-04-22T07:04:20.145 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:20.147 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T07:04:20.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.147+0000 7fcc25832640 1 -- 172.21.15.44:0/3835981325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc0006c920 msgr2=0x7fcc0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.147+0000 7fcc25832640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc0006c920 0x7fcc0006ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009d60 tx=0x7fcc18008040 comp rx=0 tx=0).stop 2024-04-22T07:04:20.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.147+0000 7fcc25832640 1 -- 172.21.15.44:0/3835981325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc201e4250 msgr2=0x7fcc201e78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.147+0000 7fcc25832640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc201e4250 0x7fcc201e78b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcc08011960 tx=0x7fcc08011e20 comp rx=0 tx=0).stop 2024-04-22T07:04:20.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.147+0000 7fcc25832640 1 -- 172.21.15.44:0/3835981325 shutdown_connections 2024-04-22T07:04:20.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.147+0000 7fcc25832640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fcc0006c920 0x7fcc0006ede0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.147+0000 7fcc25832640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fcc2015d8c0 0x7fcc201e5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.147+0000 7fcc25832640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fcc201e4250 0x7fcc201e78b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.147+0000 7fcc25832640 1 --2- 172.21.15.44:0/3835981325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c004d40 0x7fcc200c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.147+0000 7fcc25832640 1 -- 172.21.15.44:0/3835981325 >> 172.21.15.44:0/3835981325 conn(0x7fcc200735c0 msgr2=0x7fcc2015a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:20.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.148+0000 7fcc25832640 1 -- 172.21.15.44:0/3835981325 shutdown_connections 2024-04-22T07:04:20.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.148+0000 7fcc25832640 1 -- 172.21.15.44:0/3835981325 wait complete. 2024-04-22T07:04:20.157 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-22T07:04:19.711518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239940,"kb_used_data":2972,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734780,"statfs":{"total":773094113280,"available":772848414720,"internally_reserved":0,"allocated":3043328,"data_stored":1824316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.941791"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961498+0000","last_change":"2024-04-22T07:04:13.800864+0000","last_active":"2024-04-22T07:04:14.961498+0000","last_peered":"2024-04-22T07:04:14.961498+0000","last_clean":"2024-04-22T07:04:14.961498+0000","last_became_active":"2024-04-22T07:04:13.800611+0000","last_became_peered":"2024-04-22T07:04:13.800611+0000","last_unstale":"2024-04-22T07:04:14.961498+0000","last_undegraded":"2024-04-22T07:04:14.961498+0000","last_fullsized":"2024-04-22T07:04:14.961498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:55.129127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056077+0000","last_change":"2024-04-22T07:04:13.798390+0000","last_active":"2024-04-22T07:04:15.056077+0000","last_peered":"2024-04-22T07:04:15.056077+0000","last_clean":"2024-04-22T07:04:15.056077+0000","last_became_active":"2024-04-22T07:04:13.797927+0000","last_became_peered":"2024-04-22T07:04:13.797927+0000","last_unstale":"2024-04-22T07:04:15.056077+0000","last_undegraded":"2024-04-22T07:04:15.056077+0000","last_fullsized":"2024-04-22T07:04:15.056077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056425+0000","last_change":"2024-04-22T07:04:13.795288+0000","last_active":"2024-04-22T07:04:15.056425+0000","last_peered":"2024-04-22T07:04:15.056425+0000","last_clean":"2024-04-22T07:04:15.056425+0000","last_became_active":"2024-04-22T07:04:13.794704+0000","last_became_peered":"2024-04-22T07:04:13.794704+0000","last_unstale":"2024-04-22T07:04:15.056425+0000","last_undegraded":"2024-04-22T07:04:15.056425+0000","last_fullsized":"2024-04-22T07:04:15.056425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783223+0000","last_change":"2024-04-22T07:04:13.801884+0000","last_active":"2024-04-22T07:04:14.783223+0000","last_peered":"2024-04-22T07:04:14.783223+0000","last_clean":"2024-04-22T07:04:14.783223+0000","last_became_active":"2024-04-22T07:04:13.801620+0000","last_became_peered":"2024-04-22T07:04:13.801620+0000","last_unstale":"2024-04-22T07:04:14.783223+0000","last_undegraded":"2024-04-22T07:04:14.783223+0000","last_fullsized":"2024-04-22T07:04:14.783223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961151+0000","last_change":"2024-04-22T07:04:13.798670+0000","last_active":"2024-04-22T07:04:14.961151+0000","last_peered":"2024-04-22T07:04:14.961151+0000","last_clean":"2024-04-22T07:04:14.961151+0000","last_became_active":"2024-04-22T07:04:13.798384+0000","last_became_peered":"2024-04-22T07:04:13.798384+0000","last_unstale":"2024-04-22T07:04:14.961151+0000","last_undegraded":"2024-04-22T07:04:14.961151+0000","last_fullsized":"2024-04-22T07:04:14.961151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783124+0000","last_change":"2024-04-22T07:04:13.796468+0000","last_active":"2024-04-22T07:04:14.783124+0000","last_peered":"2024-04-22T07:04:14.783124+0000","last_clean":"2024-04-22T07:04:14.783124+0000","last_became_active":"2024-04-22T07:04:13.796139+0000","last_became_peered":"2024-04-22T07:04:13.796139+0000","last_unstale":"2024-04-22T07:04:14.783124+0000","last_undegraded":"2024-04-22T07:04:14.783124+0000","last_fullsized":"2024-04-22T07:04:14.783124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961066+0000","last_change":"2024-04-22T07:04:13.801449+0000","last_active":"2024-04-22T07:04:14.961066+0000","last_peered":"2024-04-22T07:04:14.961066+0000","last_clean":"2024-04-22T07:04:14.961066+0000","last_became_active":"2024-04-22T07:04:13.799374+0000","last_became_peered":"2024-04-22T07:04:13.799374+0000","last_unstale":"2024-04-22T07:04:14.961066+0000","last_undegraded":"2024-04-22T07:04:14.961066+0000","last_fullsized":"2024-04-22T07:04:14.961066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054559+0000","last_change":"2024-04-22T07:04:13.802780+0000","last_active":"2024-04-22T07:04:15.054559+0000","last_peered":"2024-04-22T07:04:15.054559+0000","last_clean":"2024-04-22T07:04:15.054559+0000","last_became_active":"2024-04-22T07:04:13.802371+0000","last_became_peered":"2024-04-22T07:04:13.802371+0000","last_unstale":"2024-04-22T07:04:15.054559+0000","last_undegraded":"2024-04-22T07:04:15.054559+0000","last_fullsized":"2024-04-22T07:04:15.054559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961532+0000","last_change":"2024-04-22T07:04:13.792483+0000","last_active":"2024-04-22T07:04:14.961532+0000","last_peered":"2024-04-22T07:04:14.961532+0000","last_clean":"2024-04-22T07:04:14.961532+0000","last_became_active":"2024-04-22T07:04:13.792198+0000","last_became_peered":"2024-04-22T07:04:13.792198+0000","last_unstale":"2024-04-22T07:04:14.961532+0000","last_undegraded":"2024-04-22T07:04:14.961532+0000","last_fullsized":"2024-04-22T07:04:14.961532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:36.943428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782040+0000","last_change":"2024-04-22T07:04:13.797379+0000","last_active":"2024-04-22T07:04:14.782040+0000","last_peered":"2024-04-22T07:04:14.782040+0000","last_clean":"2024-04-22T07:04:14.782040+0000","last_became_active":"2024-04-22T07:04:13.797029+0000","last_became_peered":"2024-04-22T07:04:13.797029+0000","last_unstale":"2024-04-22T07:04:14.782040+0000","last_undegraded":"2024-04-22T07:04:14.782040+0000","last_fullsized":"2024-04-22T07:04:14.782040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781680+0000","last_change":"2024-04-22T07:04:13.789875+0000","last_active":"2024-04-22T07:04:14.781680+0000","last_peered":"2024-04-22T07:04:14.781680+0000","last_clean":"2024-04-22T07:04:14.781680+0000","last_became_active":"2024-04-22T07:04:13.789598+0000","last_became_peered":"2024-04-22T07:04:13.789598+0000","last_unstale":"2024-04-22T07:04:14.781680+0000","last_undegraded":"2024-04-22T07:04:14.781680+0000","last_fullsized":"2024-04-22T07:04:14.781680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055156+0000","last_change":"2024-04-22T07:04:13.791103+0000","last_active":"2024-04-22T07:04:15.055156+0000","last_peered":"2024-04-22T07:04:15.055156+0000","last_clean":"2024-04-22T07:04:15.055156+0000","last_became_active":"2024-04-22T07:04:13.790819+0000","last_became_peered":"2024-04-22T07:04:13.790819+0000","last_unstale":"2024-04-22T07:04:15.055156+0000","last_undegraded":"2024-04-22T07:04:15.055156+0000","last_fullsized":"2024-04-22T07:04:15.055156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:50.632020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055402+0000","last_change":"2024-04-22T07:04:13.802407+0000","last_active":"2024-04-22T07:04:15.055402+0000","last_peered":"2024-04-22T07:04:15.055402+0000","last_clean":"2024-04-22T07:04:15.055402+0000","last_became_active":"2024-04-22T07:04:13.801785+0000","last_became_peered":"2024-04-22T07:04:13.801785+0000","last_unstale":"2024-04-22T07:04:15.055402+0000","last_undegraded":"2024-04-22T07:04:15.055402+0000","last_fullsized":"2024-04-22T07:04:15.055402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055557+0000","last_change":"2024-04-22T07:04:13.795888+0000","last_active":"2024-04-22T07:04:15.055557+0000","last_peered":"2024-04-22T07:04:15.055557+0000","last_clean":"2024-04-22T07:04:15.055557+0000","last_became_active":"2024-04-22T07:04:13.795092+0000","last_became_peered":"2024-04-22T07:04:13.795092+0000","last_unstale":"2024-04-22T07:04:15.055557+0000","last_undegraded":"2024-04-22T07:04:15.055557+0000","last_fullsized":"2024-04-22T07:04:15.055557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:54.119208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055926+0000","last_change":"2024-04-22T07:04:13.797138+0000","last_active":"2024-04-22T07:04:15.055926+0000","last_peered":"2024-04-22T07:04:15.055926+0000","last_clean":"2024-04-22T07:04:15.055926+0000","last_became_active":"2024-04-22T07:04:13.796763+0000","last_became_peered":"2024-04-22T07:04:13.796763+0000","last_unstale":"2024-04-22T07:04:15.055926+0000","last_undegraded":"2024-04-22T07:04:15.055926+0000","last_fullsized":"2024-04-22T07:04:15.055926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782465+0000","last_change":"2024-04-22T07:04:13.803869+0000","last_active":"2024-04-22T07:04:14.782465+0000","last_peered":"2024-04-22T07:04:14.782465+0000","last_clean":"2024-04-22T07:04:14.782465+0000","last_became_active":"2024-04-22T07:04:13.803451+0000","last_became_peered":"2024-04-22T07:04:13.803451+0000","last_unstale":"2024-04-22T07:04:14.782465+0000","last_undegraded":"2024-04-22T07:04:14.782465+0000","last_fullsized":"2024-04-22T07:04:14.782465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783077+0000","last_change":"2024-04-22T07:04:13.795746+0000","last_active":"2024-04-22T07:04:14.783077+0000","last_peered":"2024-04-22T07:04:14.783077+0000","last_clean":"2024-04-22T07:04:14.783077+0000","last_became_active":"2024-04-22T07:04:13.794694+0000","last_became_peered":"2024-04-22T07:04:13.794694+0000","last_unstale":"2024-04-22T07:04:14.783077+0000","last_undegraded":"2024-04-22T07:04:14.783077+0000","last_fullsized":"2024-04-22T07:04:14.783077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054829+0000","last_change":"2024-04-22T07:04:13.790670+0000","last_active":"2024-04-22T07:04:15.054829+0000","last_peered":"2024-04-22T07:04:15.054829+0000","last_clean":"2024-04-22T07:04:15.054829+0000","last_became_active":"2024-04-22T07:04:13.790104+0000","last_became_peered":"2024-04-22T07:04:13.790104+0000","last_unstale":"2024-04-22T07:04:15.054829+0000","last_undegraded":"2024-04-22T07:04:15.054829+0000","last_fullsized":"2024-04-22T07:04:15.054829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:09.325971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961464+0000","last_change":"2024-04-22T07:04:13.799016+0000","last_active":"2024-04-22T07:04:14.961464+0000","last_peered":"2024-04-22T07:04:14.961464+0000","last_clean":"2024-04-22T07:04:14.961464+0000","last_became_active":"2024-04-22T07:04:13.798716+0000","last_became_peered":"2024-04-22T07:04:13.798716+0000","last_unstale":"2024-04-22T07:04:14.961464+0000","last_undegraded":"2024-04-22T07:04:14.961464+0000","last_fullsized":"2024-04-22T07:04:14.961464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:36.609382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781586+0000","last_change":"2024-04-22T07:04:13.797296+0000","last_active":"2024-04-22T07:04:14.781586+0000","last_peered":"2024-04-22T07:04:14.781586+0000","last_clean":"2024-04-22T07:04:14.781586+0000","last_became_active":"2024-04-22T07:04:13.796997+0000","last_became_peered":"2024-04-22T07:04:13.796997+0000","last_unstale":"2024-04-22T07:04:14.781586+0000","last_undegraded":"2024-04-22T07:04:14.781586+0000","last_fullsized":"2024-04-22T07:04:14.781586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055994+0000","last_change":"2024-04-22T07:04:13.799675+0000","last_active":"2024-04-22T07:04:15.055994+0000","last_peered":"2024-04-22T07:04:15.055994+0000","last_clean":"2024-04-22T07:04:15.055994+0000","last_became_active":"2024-04-22T07:04:13.799411+0000","last_became_peered":"2024-04-22T07:04:13.799411+0000","last_unstale":"2024-04-22T07:04:15.055994+0000","last_undegraded":"2024-04-22T07:04:15.055994+0000","last_fullsized":"2024-04-22T07:04:15.055994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783058+0000","last_change":"2024-04-22T07:04:13.796128+0000","last_active":"2024-04-22T07:04:14.783058+0000","last_peered":"2024-04-22T07:04:14.783058+0000","last_clean":"2024-04-22T07:04:14.783058+0000","last_became_active":"2024-04-22T07:04:13.795330+0000","last_became_peered":"2024-04-22T07:04:13.795330+0000","last_unstale":"2024-04-22T07:04:14.783058+0000","last_undegraded":"2024-04-22T07:04:14.783058+0000","last_fullsized":"2024-04-22T07:04:14.783058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055294+0000","last_change":"2024-04-22T07:04:13.800458+0000","last_active":"2024-04-22T07:04:15.055294+0000","last_peered":"2024-04-22T07:04:15.055294+0000","last_clean":"2024-04-22T07:04:15.055294+0000","last_became_active":"2024-04-22T07:04:13.800171+0000","last_became_peered":"2024-04-22T07:04:13.800171+0000","last_unstale":"2024-04-22T07:04:15.055294+0000","last_undegraded":"2024-04-22T07:04:15.055294+0000","last_fullsized":"2024-04-22T07:04:15.055294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782518+0000","last_change":"2024-04-22T07:04:13.802472+0000","last_active":"2024-04-22T07:04:14.782518+0000","last_peered":"2024-04-22T07:04:14.782518+0000","last_clean":"2024-04-22T07:04:14.782518+0000","last_became_active":"2024-04-22T07:04:13.802201+0000","last_became_peered":"2024-04-22T07:04:13.802201+0000","last_unstale":"2024-04-22T07:04:14.782518+0000","last_undegraded":"2024-04-22T07:04:14.782518+0000","last_fullsized":"2024-04-22T07:04:14.782518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055335+0000","last_change":"2024-04-22T07:04:13.795055+0000","last_active":"2024-04-22T07:04:15.055335+0000","last_peered":"2024-04-22T07:04:15.055335+0000","last_clean":"2024-04-22T07:04:15.055335+0000","last_became_active":"2024-04-22T07:04:13.794145+0000","last_became_peered":"2024-04-22T07:04:13.794145+0000","last_unstale":"2024-04-22T07:04:15.055335+0000","last_undegraded":"2024-04-22T07:04:15.055335+0000","last_fullsized":"2024-04-22T07:04:15.055335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782104+0000","last_change":"2024-04-22T07:04:13.799772+0000","last_active":"2024-04-22T07:04:14.782104+0000","last_peered":"2024-04-22T07:04:14.782104+0000","last_clean":"2024-04-22T07:04:14.782104+0000","last_became_active":"2024-04-22T07:04:13.799504+0000","last_became_peered":"2024-04-22T07:04:13.799504+0000","last_unstale":"2024-04-22T07:04:14.782104+0000","last_undegraded":"2024-04-22T07:04:14.782104+0000","last_fullsized":"2024-04-22T07:04:14.782104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056133+0000","last_change":"2024-04-22T07:04:13.804604+0000","last_active":"2024-04-22T07:04:15.056133+0000","last_peered":"2024-04-22T07:04:15.056133+0000","last_clean":"2024-04-22T07:04:15.056133+0000","last_became_active":"2024-04-22T07:04:13.803594+0000","last_became_peered":"2024-04-22T07:04:13.803594+0000","last_unstale":"2024-04-22T07:04:15.056133+0000","last_undegraded":"2024-04-22T07:04:15.056133+0000","last_fullsized":"2024-04-22T07:04:15.056133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055262+0000","last_change":"2024-04-22T07:04:13.795947+0000","last_active":"2024-04-22T07:04:15.055262+0000","last_peered":"2024-04-22T07:04:15.055262+0000","last_clean":"2024-04-22T07:04:15.055262+0000","last_became_active":"2024-04-22T07:04:13.795223+0000","last_became_peered":"2024-04-22T07:04:13.795223+0000","last_unstale":"2024-04-22T07:04:15.055262+0000","last_undegraded":"2024-04-22T07:04:15.055262+0000","last_fullsized":"2024-04-22T07:04:15.055262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056195+0000","last_change":"2024-04-22T07:04:13.804225+0000","last_active":"2024-04-22T07:04:15.056195+0000","last_peered":"2024-04-22T07:04:15.056195+0000","last_clean":"2024-04-22T07:04:15.056195+0000","last_became_active":"2024-04-22T07:04:13.803920+0000","last_became_peered":"2024-04-22T07:04:13.803920+0000","last_unstale":"2024-04-22T07:04:15.056195+0000","last_undegraded":"2024-04-22T07:04:15.056195+0000","last_fullsized":"2024-04-22T07:04:15.056195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783228+0000","last_change":"2024-04-22T07:04:13.796357+0000","last_active":"2024-04-22T07:04:14.783228+0000","last_peered":"2024-04-22T07:04:14.783228+0000","last_clean":"2024-04-22T07:04:14.783228+0000","last_became_active":"2024-04-22T07:04:13.795784+0000","last_became_peered":"2024-04-22T07:04:13.795784+0000","last_unstale":"2024-04-22T07:04:14.783228+0000","last_undegraded":"2024-04-22T07:04:14.783228+0000","last_fullsized":"2024-04-22T07:04:14.783228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054855+0000","last_change":"2024-04-22T07:04:13.784337+0000","last_active":"2024-04-22T07:04:15.054855+0000","last_peered":"2024-04-22T07:04:15.054855+0000","last_clean":"2024-04-22T07:04:15.054855+0000","last_became_active":"2024-04-22T07:04:13.783914+0000","last_became_peered":"2024-04-22T07:04:13.783914+0000","last_unstale":"2024-04-22T07:04:15.054855+0000","last_undegraded":"2024-04-22T07:04:15.054855+0000","last_fullsized":"2024-04-22T07:04:15.054855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:43:41.211395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781515+0000","last_change":"2024-04-22T07:04:13.806969+0000","last_active":"2024-04-22T07:04:14.781515+0000","last_peered":"2024-04-22T07:04:14.781515+0000","last_clean":"2024-04-22T07:04:14.781515+0000","last_became_active":"2024-04-22T07:04:13.806715+0000","last_became_peered":"2024-04-22T07:04:13.806715+0000","last_unstale":"2024-04-22T07:04:14.781515+0000","last_undegraded":"2024-04-22T07:04:14.781515+0000","last_fullsized":"2024-04-22T07:04:14.781515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054895+0000","last_change":"2024-04-22T07:04:13.794896+0000","last_active":"2024-04-22T07:04:15.054895+0000","last_peered":"2024-04-22T07:04:15.054895+0000","last_clean":"2024-04-22T07:04:15.054895+0000","last_became_active":"2024-04-22T07:04:13.794605+0000","last_became_peered":"2024-04-22T07:04:13.794605+0000","last_unstale":"2024-04-22T07:04:15.054895+0000","last_undegraded":"2024-04-22T07:04:15.054895+0000","last_fullsized":"2024-04-22T07:04:15.054895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:02.613130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055503+0000","last_change":"2024-04-22T07:04:12.786738+0000","last_active":"2024-04-22T07:04:15.055503+0000","last_peered":"2024-04-22T07:04:15.055503+0000","last_clean":"2024-04-22T07:04:15.055503+0000","last_became_active":"2024-04-22T07:04:12.786413+0000","last_became_peered":"2024-04-22T07:04:12.786413+0000","last_unstale":"2024-04-22T07:04:15.055503+0000","last_undegraded":"2024-04-22T07:04:15.055503+0000","last_fullsized":"2024-04-22T07:04:15.055503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056196+0000","last_change":"2024-04-22T07:04:13.800525+0000","last_active":"2024-04-22T07:04:15.056196+0000","last_peered":"2024-04-22T07:04:15.056196+0000","last_clean":"2024-04-22T07:04:15.056196+0000","last_became_active":"2024-04-22T07:04:13.800254+0000","last_became_peered":"2024-04-22T07:04:13.800254+0000","last_unstale":"2024-04-22T07:04:15.056196+0000","last_undegraded":"2024-04-22T07:04:15.056196+0000","last_fullsized":"2024-04-22T07:04:15.056196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781988+0000","last_change":"2024-04-22T07:04:12.786726+0000","last_active":"2024-04-22T07:04:14.781988+0000","last_peered":"2024-04-22T07:04:14.781988+0000","last_clean":"2024-04-22T07:04:14.781988+0000","last_became_active":"2024-04-22T07:04:12.786444+0000","last_became_peered":"2024-04-22T07:04:12.786444+0000","last_unstale":"2024-04-22T07:04:14.781988+0000","last_undegraded":"2024-04-22T07:04:14.781988+0000","last_fullsized":"2024-04-22T07:04:14.781988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055394+0000","last_change":"2024-04-22T07:04:12.787357+0000","last_active":"2024-04-22T07:04:15.055394+0000","last_peered":"2024-04-22T07:04:15.055394+0000","last_clean":"2024-04-22T07:04:15.055394+0000","last_became_active":"2024-04-22T07:04:12.787066+0000","last_became_peered":"2024-04-22T07:04:12.787066+0000","last_unstale":"2024-04-22T07:04:15.055394+0000","last_undegraded":"2024-04-22T07:04:15.055394+0000","last_fullsized":"2024-04-22T07:04:15.055394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054713+0000","last_change":"2024-04-22T07:04:13.807697+0000","last_active":"2024-04-22T07:04:15.054713+0000","last_peered":"2024-04-22T07:04:15.054713+0000","last_clean":"2024-04-22T07:04:15.054713+0000","last_became_active":"2024-04-22T07:04:13.807327+0000","last_became_peered":"2024-04-22T07:04:13.807327+0000","last_unstale":"2024-04-22T07:04:15.054713+0000","last_undegraded":"2024-04-22T07:04:15.054713+0000","last_fullsized":"2024-04-22T07:04:15.054713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783313+0000","last_change":"2024-04-22T07:04:12.787783+0000","last_active":"2024-04-22T07:04:14.783313+0000","last_peered":"2024-04-22T07:04:14.783313+0000","last_clean":"2024-04-22T07:04:14.783313+0000","last_became_active":"2024-04-22T07:04:12.786765+0000","last_became_peered":"2024-04-22T07:04:12.786765+0000","last_unstale":"2024-04-22T07:04:14.783313+0000","last_undegraded":"2024-04-22T07:04:14.783313+0000","last_fullsized":"2024-04-22T07:04:14.783313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055740+0000","last_change":"2024-04-22T07:04:13.804931+0000","last_active":"2024-04-22T07:04:15.055740+0000","last_peered":"2024-04-22T07:04:15.055740+0000","last_clean":"2024-04-22T07:04:15.055740+0000","last_became_active":"2024-04-22T07:04:13.804629+0000","last_became_peered":"2024-04-22T07:04:13.804629+0000","last_unstale":"2024-04-22T07:04:15.055740+0000","last_undegraded":"2024-04-22T07:04:15.055740+0000","last_fullsized":"2024-04-22T07:04:15.055740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781808+0000","last_change":"2024-04-22T07:04:12.789279+0000","last_active":"2024-04-22T07:04:14.781808+0000","last_peered":"2024-04-22T07:04:14.781808+0000","last_clean":"2024-04-22T07:04:14.781808+0000","last_became_active":"2024-04-22T07:04:12.787705+0000","last_became_peered":"2024-04-22T07:04:12.787705+0000","last_unstale":"2024-04-22T07:04:14.781808+0000","last_undegraded":"2024-04-22T07:04:14.781808+0000","last_fullsized":"2024-04-22T07:04:14.781808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782812+0000","last_change":"2024-04-22T07:04:13.794292+0000","last_active":"2024-04-22T07:04:14.782812+0000","last_peered":"2024-04-22T07:04:14.782812+0000","last_clean":"2024-04-22T07:04:14.782812+0000","last_became_active":"2024-04-22T07:04:13.793986+0000","last_became_peered":"2024-04-22T07:04:13.793986+0000","last_unstale":"2024-04-22T07:04:14.782812+0000","last_undegraded":"2024-04-22T07:04:14.782812+0000","last_fullsized":"2024-04-22T07:04:14.782812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782990+0000","last_change":"2024-04-22T07:04:12.787064+0000","last_active":"2024-04-22T07:04:14.782990+0000","last_peered":"2024-04-22T07:04:14.782990+0000","last_clean":"2024-04-22T07:04:14.782990+0000","last_became_active":"2024-04-22T07:04:12.786430+0000","last_became_peered":"2024-04-22T07:04:12.786430+0000","last_unstale":"2024-04-22T07:04:14.782990+0000","last_undegraded":"2024-04-22T07:04:14.782990+0000","last_fullsized":"2024-04-22T07:04:14.782990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056732+0000","last_change":"2024-04-22T07:04:13.801337+0000","last_active":"2024-04-22T07:04:15.056732+0000","last_peered":"2024-04-22T07:04:15.056732+0000","last_clean":"2024-04-22T07:04:15.056732+0000","last_became_active":"2024-04-22T07:04:13.800916+0000","last_became_peered":"2024-04-22T07:04:13.800916+0000","last_unstale":"2024-04-22T07:04:15.056732+0000","last_undegraded":"2024-04-22T07:04:15.056732+0000","last_fullsized":"2024-04-22T07:04:15.056732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056192+0000","last_change":"2024-04-22T07:04:12.787741+0000","last_active":"2024-04-22T07:04:15.056192+0000","last_peered":"2024-04-22T07:04:15.056192+0000","last_clean":"2024-04-22T07:04:15.056192+0000","last_became_active":"2024-04-22T07:04:12.787405+0000","last_became_peered":"2024-04-22T07:04:12.787405+0000","last_unstale":"2024-04-22T07:04:15.056192+0000","last_undegraded":"2024-04-22T07:04:15.056192+0000","last_fullsized":"2024-04-22T07:04:15.056192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055184+0000","last_change":"2024-04-22T07:04:13.795502+0000","last_active":"2024-04-22T07:04:15.055184+0000","last_peered":"2024-04-22T07:04:15.055184+0000","last_clean":"2024-04-22T07:04:15.055184+0000","last_became_active":"2024-04-22T07:04:13.795199+0000","last_became_peered":"2024-04-22T07:04:13.795199+0000","last_unstale":"2024-04-22T07:04:15.055184+0000","last_undegraded":"2024-04-22T07:04:15.055184+0000","last_fullsized":"2024-04-22T07:04:15.055184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:07.780008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781679+0000","last_change":"2024-04-22T07:04:12.788572+0000","last_active":"2024-04-22T07:04:14.781679+0000","last_peered":"2024-04-22T07:04:14.781679+0000","last_clean":"2024-04-22T07:04:14.781679+0000","last_became_active":"2024-04-22T07:04:12.788286+0000","last_became_peered":"2024-04-22T07:04:12.788286+0000","last_unstale":"2024-04-22T07:04:14.781679+0000","last_undegraded":"2024-04-22T07:04:14.781679+0000","last_fullsized":"2024-04-22T07:04:14.781679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961160+0000","last_change":"2024-04-22T07:04:13.804435+0000","last_active":"2024-04-22T07:04:14.961160+0000","last_peered":"2024-04-22T07:04:14.961160+0000","last_clean":"2024-04-22T07:04:14.961160+0000","last_became_active":"2024-04-22T07:04:13.804170+0000","last_became_peered":"2024-04-22T07:04:13.804170+0000","last_unstale":"2024-04-22T07:04:14.961160+0000","last_undegraded":"2024-04-22T07:04:14.961160+0000","last_fullsized":"2024-04-22T07:04:14.961160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056278+0000","last_change":"2024-04-22T07:04:12.789905+0000","last_active":"2024-04-22T07:04:15.056278+0000","last_peered":"2024-04-22T07:04:15.056278+0000","last_clean":"2024-04-22T07:04:15.056278+0000","last_became_active":"2024-04-22T07:04:12.789503+0000","last_became_peered":"2024-04-22T07:04:12.789503+0000","last_unstale":"2024-04-22T07:04:15.056278+0000","last_undegraded":"2024-04-22T07:04:15.056278+0000","last_fullsized":"2024-04-22T07:04:15.056278+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055741+0000","last_change":"2024-04-22T07:04:13.801490+0000","last_active":"2024-04-22T07:04:15.055741+0000","last_peered":"2024-04-22T07:04:15.055741+0000","last_clean":"2024-04-22T07:04:15.055741+0000","last_became_active":"2024-04-22T07:04:13.799356+0000","last_became_peered":"2024-04-22T07:04:13.799356+0000","last_unstale":"2024-04-22T07:04:15.055741+0000","last_undegraded":"2024-04-22T07:04:15.055741+0000","last_fullsized":"2024-04-22T07:04:15.055741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:32:04.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783198+0000","last_change":"2024-04-22T07:04:12.781089+0000","last_active":"2024-04-22T07:04:14.783198+0000","last_peered":"2024-04-22T07:04:14.783198+0000","last_clean":"2024-04-22T07:04:14.783198+0000","last_became_active":"2024-04-22T07:04:12.780827+0000","last_became_peered":"2024-04-22T07:04:12.780827+0000","last_unstale":"2024-04-22T07:04:14.783198+0000","last_undegraded":"2024-04-22T07:04:14.783198+0000","last_fullsized":"2024-04-22T07:04:14.783198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961202+0000","last_change":"2024-04-22T07:04:13.801375+0000","last_active":"2024-04-22T07:04:14.961202+0000","last_peered":"2024-04-22T07:04:14.961202+0000","last_clean":"2024-04-22T07:04:14.961202+0000","last_became_active":"2024-04-22T07:04:13.799488+0000","last_became_peered":"2024-04-22T07:04:13.799488+0000","last_unstale":"2024-04-22T07:04:14.961202+0000","last_undegraded":"2024-04-22T07:04:14.961202+0000","last_fullsized":"2024-04-22T07:04:14.961202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781576+0000","last_change":"2024-04-22T07:04:12.790758+0000","last_active":"2024-04-22T07:04:14.781576+0000","last_peered":"2024-04-22T07:04:14.781576+0000","last_clean":"2024-04-22T07:04:14.781576+0000","last_became_active":"2024-04-22T07:04:12.790444+0000","last_became_peered":"2024-04-22T07:04:12.790444+0000","last_unstale":"2024-04-22T07:04:14.781576+0000","last_undegraded":"2024-04-22T07:04:14.781576+0000","last_fullsized":"2024-04-22T07:04:14.781576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782297+0000","last_change":"2024-04-22T07:04:13.805415+0000","last_active":"2024-04-22T07:04:14.782297+0000","last_peered":"2024-04-22T07:04:14.782297+0000","last_clean":"2024-04-22T07:04:14.782297+0000","last_became_active":"2024-04-22T07:04:13.805115+0000","last_became_peered":"2024-04-22T07:04:13.805115+0000","last_unstale":"2024-04-22T07:04:14.782297+0000","last_undegraded":"2024-04-22T07:04:14.782297+0000","last_fullsized":"2024-04-22T07:04:14.782297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054450+0000","last_change":"2024-04-22T07:04:12.789877+0000","last_active":"2024-04-22T07:04:15.054450+0000","last_peered":"2024-04-22T07:04:15.054450+0000","last_clean":"2024-04-22T07:04:15.054450+0000","last_became_active":"2024-04-22T07:04:12.789546+0000","last_became_peered":"2024-04-22T07:04:12.789546+0000","last_unstale":"2024-04-22T07:04:15.054450+0000","last_undegraded":"2024-04-22T07:04:15.054450+0000","last_fullsized":"2024-04-22T07:04:15.054450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:04.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961082+0000","last_change":"2024-04-22T07:04:13.801803+0000","last_active":"2024-04-22T07:04:14.961082+0000","last_peered":"2024-04-22T07:04:14.961082+0000","last_clean":"2024-04-22T07:04:14.961082+0000","last_became_active":"2024-04-22T07:04:13.801362+0000","last_became_peered":"2024-04-22T07:04:13.801362+0000","last_unstale":"2024-04-22T07:04:14.961082+0000","last_undegraded":"2024-04-22T07:04:14.961082+0000","last_fullsized":"2024-04-22T07:04:14.961082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055197+0000","last_change":"2024-04-22T07:04:12.787917+0000","last_active":"2024-04-22T07:04:15.055197+0000","last_peered":"2024-04-22T07:04:15.055197+0000","last_clean":"2024-04-22T07:04:15.055197+0000","last_became_active":"2024-04-22T07:04:12.787636+0000","last_became_peered":"2024-04-22T07:04:12.787636+0000","last_unstale":"2024-04-22T07:04:15.055197+0000","last_undegraded":"2024-04-22T07:04:15.055197+0000","last_fullsized":"2024-04-22T07:04:15.055197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961456+0000","last_change":"2024-04-22T07:04:13.802786+0000","last_active":"2024-04-22T07:04:14.961456+0000","last_peered":"2024-04-22T07:04:14.961456+0000","last_clean":"2024-04-22T07:04:14.961456+0000","last_became_active":"2024-04-22T07:04:13.801299+0000","last_became_peered":"2024-04-22T07:04:13.801299+0000","last_unstale":"2024-04-22T07:04:14.961456+0000","last_undegraded":"2024-04-22T07:04:14.961456+0000","last_fullsized":"2024-04-22T07:04:14.961456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:28.362520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054920+0000","last_change":"2024-04-22T07:04:12.788593+0000","last_active":"2024-04-22T07:04:15.054920+0000","last_peered":"2024-04-22T07:04:15.054920+0000","last_clean":"2024-04-22T07:04:15.054920+0000","last_became_active":"2024-04-22T07:04:12.788314+0000","last_became_peered":"2024-04-22T07:04:12.788314+0000","last_unstale":"2024-04-22T07:04:15.054920+0000","last_undegraded":"2024-04-22T07:04:15.054920+0000","last_fullsized":"2024-04-22T07:04:15.054920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:11:27.081266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961608+0000","last_change":"2024-04-22T07:04:13.800245+0000","last_active":"2024-04-22T07:04:14.961608+0000","last_peered":"2024-04-22T07:04:14.961608+0000","last_clean":"2024-04-22T07:04:14.961608+0000","last_became_active":"2024-04-22T07:04:13.799970+0000","last_became_peered":"2024-04-22T07:04:13.799970+0000","last_unstale":"2024-04-22T07:04:14.961608+0000","last_undegraded":"2024-04-22T07:04:14.961608+0000","last_fullsized":"2024-04-22T07:04:14.961608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:13.558356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056239+0000","last_change":"2024-04-22T07:04:12.790003+0000","last_active":"2024-04-22T07:04:15.056239+0000","last_peered":"2024-04-22T07:04:15.056239+0000","last_clean":"2024-04-22T07:04:15.056239+0000","last_became_active":"2024-04-22T07:04:12.789708+0000","last_became_peered":"2024-04-22T07:04:12.789708+0000","last_unstale":"2024-04-22T07:04:15.056239+0000","last_undegraded":"2024-04-22T07:04:15.056239+0000","last_fullsized":"2024-04-22T07:04:15.056239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783345+0000","last_change":"2024-04-22T07:04:13.795082+0000","last_active":"2024-04-22T07:04:14.783345+0000","last_peered":"2024-04-22T07:04:14.783345+0000","last_clean":"2024-04-22T07:04:14.783345+0000","last_became_active":"2024-04-22T07:04:13.794793+0000","last_became_peered":"2024-04-22T07:04:13.794793+0000","last_unstale":"2024-04-22T07:04:14.783345+0000","last_undegraded":"2024-04-22T07:04:14.783345+0000","last_fullsized":"2024-04-22T07:04:14.783345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054608+0000","last_change":"2024-04-22T07:04:12.789252+0000","last_active":"2024-04-22T07:04:15.054608+0000","last_peered":"2024-04-22T07:04:15.054608+0000","last_clean":"2024-04-22T07:04:15.054608+0000","last_became_active":"2024-04-22T07:04:12.788946+0000","last_became_peered":"2024-04-22T07:04:12.788946+0000","last_unstale":"2024-04-22T07:04:15.054608+0000","last_undegraded":"2024-04-22T07:04:15.054608+0000","last_fullsized":"2024-04-22T07:04:15.054608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:36:13.428703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961272+0000","last_change":"2024-04-22T07:04:13.801250+0000","last_active":"2024-04-22T07:04:14.961272+0000","last_peered":"2024-04-22T07:04:14.961272+0000","last_clean":"2024-04-22T07:04:14.961272+0000","last_became_active":"2024-04-22T07:04:13.799143+0000","last_became_peered":"2024-04-22T07:04:13.799143+0000","last_unstale":"2024-04-22T07:04:14.961272+0000","last_undegraded":"2024-04-22T07:04:14.961272+0000","last_fullsized":"2024-04-22T07:04:14.961272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960952+0000","last_change":"2024-04-22T07:04:03.688462+0000","last_active":"2024-04-22T07:04:14.960952+0000","last_peered":"2024-04-22T07:04:14.960952+0000","last_clean":"2024-04-22T07:04:14.960952+0000","last_became_active":"2024-04-22T07:04:03.688191+0000","last_became_peered":"2024-04-22T07:04:03.688191+0000","last_unstale":"2024-04-22T07:04:14.960952+0000","last_undegraded":"2024-04-22T07:04:14.960952+0000","last_fullsized":"2024-04-22T07:04:14.960952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.464330+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.053999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782468+0000","last_change":"2024-04-22T07:04:12.783751+0000","last_active":"2024-04-22T07:04:14.782468+0000","last_peered":"2024-04-22T07:04:14.782468+0000","last_clean":"2024-04-22T07:04:14.782468+0000","last_became_active":"2024-04-22T07:04:12.783460+0000","last_became_peered":"2024-04-22T07:04:12.783460+0000","last_unstale":"2024-04-22T07:04:14.782468+0000","last_undegraded":"2024-04-22T07:04:14.782468+0000","last_fullsized":"2024-04-22T07:04:14.782468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054881+0000","last_change":"2024-04-22T07:04:13.791961+0000","last_active":"2024-04-22T07:04:15.054881+0000","last_peered":"2024-04-22T07:04:15.054881+0000","last_clean":"2024-04-22T07:04:15.054881+0000","last_became_active":"2024-04-22T07:04:13.790591+0000","last_became_peered":"2024-04-22T07:04:13.790591+0000","last_unstale":"2024-04-22T07:04:15.054881+0000","last_undegraded":"2024-04-22T07:04:15.054881+0000","last_fullsized":"2024-04-22T07:04:15.054881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960999+0000","last_change":"2024-04-22T07:04:12.787268+0000","last_active":"2024-04-22T07:04:14.960999+0000","last_peered":"2024-04-22T07:04:14.960999+0000","last_clean":"2024-04-22T07:04:14.960999+0000","last_became_active":"2024-04-22T07:04:12.786991+0000","last_became_peered":"2024-04-22T07:04:12.786991+0000","last_unstale":"2024-04-22T07:04:14.960999+0000","last_undegraded":"2024-04-22T07:04:14.960999+0000","last_fullsized":"2024-04-22T07:04:14.960999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054726+0000","last_change":"2024-04-22T07:04:13.800243+0000","last_active":"2024-04-22T07:04:15.054726+0000","last_peered":"2024-04-22T07:04:15.054726+0000","last_clean":"2024-04-22T07:04:15.054726+0000","last_became_active":"2024-04-22T07:04:13.799917+0000","last_became_peered":"2024-04-22T07:04:13.799917+0000","last_unstale":"2024-04-22T07:04:15.054726+0000","last_undegraded":"2024-04-22T07:04:15.054726+0000","last_fullsized":"2024-04-22T07:04:15.054726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:54.958916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054937+0000","last_change":"2024-04-22T07:04:12.787893+0000","last_active":"2024-04-22T07:04:15.054937+0000","last_peered":"2024-04-22T07:04:15.054937+0000","last_clean":"2024-04-22T07:04:15.054937+0000","last_became_active":"2024-04-22T07:04:12.787370+0000","last_became_peered":"2024-04-22T07:04:12.787370+0000","last_unstale":"2024-04-22T07:04:15.054937+0000","last_undegraded":"2024-04-22T07:04:15.054937+0000","last_fullsized":"2024-04-22T07:04:15.054937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:52:03.220670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055952+0000","last_change":"2024-04-22T07:04:13.795454+0000","last_active":"2024-04-22T07:04:15.055952+0000","last_peered":"2024-04-22T07:04:15.055952+0000","last_clean":"2024-04-22T07:04:15.055952+0000","last_became_active":"2024-04-22T07:04:13.795138+0000","last_became_peered":"2024-04-22T07:04:13.795138+0000","last_unstale":"2024-04-22T07:04:15.055952+0000","last_undegraded":"2024-04-22T07:04:15.055952+0000","last_fullsized":"2024-04-22T07:04:15.055952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783273+0000","last_change":"2024-04-22T07:04:12.786891+0000","last_active":"2024-04-22T07:04:14.783273+0000","last_peered":"2024-04-22T07:04:14.783273+0000","last_clean":"2024-04-22T07:04:14.783273+0000","last_became_active":"2024-04-22T07:04:12.786557+0000","last_became_peered":"2024-04-22T07:04:12.786557+0000","last_unstale":"2024-04-22T07:04:14.783273+0000","last_undegraded":"2024-04-22T07:04:14.783273+0000","last_fullsized":"2024-04-22T07:04:14.783273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055054+0000","last_change":"2024-04-22T07:04:13.798678+0000","last_active":"2024-04-22T07:04:15.055054+0000","last_peered":"2024-04-22T07:04:15.055054+0000","last_clean":"2024-04-22T07:04:15.055054+0000","last_became_active":"2024-04-22T07:04:13.798369+0000","last_became_peered":"2024-04-22T07:04:13.798369+0000","last_unstale":"2024-04-22T07:04:15.055054+0000","last_undegraded":"2024-04-22T07:04:15.055054+0000","last_fullsized":"2024-04-22T07:04:15.055054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:37.919431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055841+0000","last_change":"2024-04-22T07:04:12.791897+0000","last_active":"2024-04-22T07:04:15.055841+0000","last_peered":"2024-04-22T07:04:15.055841+0000","last_clean":"2024-04-22T07:04:15.055841+0000","last_became_active":"2024-04-22T07:04:12.791580+0000","last_became_peered":"2024-04-22T07:04:12.791580+0000","last_unstale":"2024-04-22T07:04:15.055841+0000","last_undegraded":"2024-04-22T07:04:15.055841+0000","last_fullsized":"2024-04-22T07:04:15.055841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782942+0000","last_change":"2024-04-22T07:04:13.803516+0000","last_active":"2024-04-22T07:04:14.782942+0000","last_peered":"2024-04-22T07:04:14.782942+0000","last_clean":"2024-04-22T07:04:14.782942+0000","last_became_active":"2024-04-22T07:04:13.803230+0000","last_became_peered":"2024-04-22T07:04:13.803230+0000","last_unstale":"2024-04-22T07:04:14.782942+0000","last_undegraded":"2024-04-22T07:04:14.782942+0000","last_fullsized":"2024-04-22T07:04:14.782942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961432+0000","last_change":"2024-04-22T07:04:12.789499+0000","last_active":"2024-04-22T07:04:14.961432+0000","last_peered":"2024-04-22T07:04:14.961432+0000","last_clean":"2024-04-22T07:04:14.961432+0000","last_became_active":"2024-04-22T07:04:12.789229+0000","last_became_peered":"2024-04-22T07:04:12.789229+0000","last_unstale":"2024-04-22T07:04:14.961432+0000","last_undegraded":"2024-04-22T07:04:14.961432+0000","last_fullsized":"2024-04-22T07:04:14.961432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:53.955113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055856+0000","last_change":"2024-04-22T07:04:13.797969+0000","last_active":"2024-04-22T07:04:15.055856+0000","last_peered":"2024-04-22T07:04:15.055856+0000","last_clean":"2024-04-22T07:04:15.055856+0000","last_became_active":"2024-04-22T07:04:13.797662+0000","last_became_peered":"2024-04-22T07:04:13.797662+0000","last_unstale":"2024-04-22T07:04:15.055856+0000","last_undegraded":"2024-04-22T07:04:15.055856+0000","last_fullsized":"2024-04-22T07:04:15.055856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056169+0000","last_change":"2024-04-22T07:04:12.791787+0000","last_active":"2024-04-22T07:04:15.056169+0000","last_peered":"2024-04-22T07:04:15.056169+0000","last_clean":"2024-04-22T07:04:15.056169+0000","last_became_active":"2024-04-22T07:04:12.791460+0000","last_became_peered":"2024-04-22T07:04:12.791460+0000","last_unstale":"2024-04-22T07:04:15.056169+0000","last_undegraded":"2024-04-22T07:04:15.056169+0000","last_fullsized":"2024-04-22T07:04:15.056169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055451+0000","last_change":"2024-04-22T07:04:13.797755+0000","last_active":"2024-04-22T07:04:15.055451+0000","last_peered":"2024-04-22T07:04:15.055451+0000","last_clean":"2024-04-22T07:04:15.055451+0000","last_became_active":"2024-04-22T07:04:13.797400+0000","last_became_peered":"2024-04-22T07:04:13.797400+0000","last_unstale":"2024-04-22T07:04:15.055451+0000","last_undegraded":"2024-04-22T07:04:15.055451+0000","last_fullsized":"2024-04-22T07:04:15.055451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781905+0000","last_change":"2024-04-22T07:04:12.790458+0000","last_active":"2024-04-22T07:04:14.781905+0000","last_peered":"2024-04-22T07:04:14.781905+0000","last_clean":"2024-04-22T07:04:14.781905+0000","last_became_active":"2024-04-22T07:04:12.790209+0000","last_became_peered":"2024-04-22T07:04:12.790209+0000","last_unstale":"2024-04-22T07:04:14.781905+0000","last_undegraded":"2024-04-22T07:04:14.781905+0000","last_fullsized":"2024-04-22T07:04:14.781905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056058+0000","last_change":"2024-04-22T07:04:13.796176+0000","last_active":"2024-04-22T07:04:15.056058+0000","last_peered":"2024-04-22T07:04:15.056058+0000","last_clean":"2024-04-22T07:04:15.056058+0000","last_became_active":"2024-04-22T07:04:13.795325+0000","last_became_peered":"2024-04-22T07:04:13.795325+0000","last_unstale":"2024-04-22T07:04:15.056058+0000","last_undegraded":"2024-04-22T07:04:15.056058+0000","last_fullsized":"2024-04-22T07:04:15.056058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960859+0000","last_change":"2024-04-22T07:04:12.791157+0000","last_active":"2024-04-22T07:04:14.960859+0000","last_peered":"2024-04-22T07:04:14.960859+0000","last_clean":"2024-04-22T07:04:14.960859+0000","last_became_active":"2024-04-22T07:04:12.790879+0000","last_became_peered":"2024-04-22T07:04:12.790879+0000","last_unstale":"2024-04-22T07:04:14.960859+0000","last_undegraded":"2024-04-22T07:04:14.960859+0000","last_fullsized":"2024-04-22T07:04:14.960859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054928+0000","last_change":"2024-04-22T07:04:13.796587+0000","last_active":"2024-04-22T07:04:15.054928+0000","last_peered":"2024-04-22T07:04:15.054928+0000","last_clean":"2024-04-22T07:04:15.054928+0000","last_became_active":"2024-04-22T07:04:13.796305+0000","last_became_peered":"2024-04-22T07:04:13.796305+0000","last_unstale":"2024-04-22T07:04:15.054928+0000","last_undegraded":"2024-04-22T07:04:15.054928+0000","last_fullsized":"2024-04-22T07:04:15.054928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:29.793237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055857+0000","last_change":"2024-04-22T07:04:12.784359+0000","last_active":"2024-04-22T07:04:15.055857+0000","last_peered":"2024-04-22T07:04:15.055857+0000","last_clean":"2024-04-22T07:04:15.055857+0000","last_became_active":"2024-04-22T07:04:12.784076+0000","last_became_peered":"2024-04-22T07:04:12.784076+0000","last_unstale":"2024-04-22T07:04:15.055857+0000","last_undegraded":"2024-04-22T07:04:15.055857+0000","last_fullsized":"2024-04-22T07:04:15.055857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054875+0000","last_change":"2024-04-22T07:04:13.802658+0000","last_active":"2024-04-22T07:04:15.054875+0000","last_peered":"2024-04-22T07:04:15.054875+0000","last_clean":"2024-04-22T07:04:15.054875+0000","last_became_active":"2024-04-22T07:04:13.802059+0000","last_became_peered":"2024-04-22T07:04:13.802059+0000","last_unstale":"2024-04-22T07:04:15.054875+0000","last_undegraded":"2024-04-22T07:04:15.054875+0000","last_fullsized":"2024-04-22T07:04:15.054875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055829+0000","last_change":"2024-04-22T07:04:12.788090+0000","last_active":"2024-04-22T07:04:15.055829+0000","last_peered":"2024-04-22T07:04:15.055829+0000","last_clean":"2024-04-22T07:04:15.055829+0000","last_became_active":"2024-04-22T07:04:12.787818+0000","last_became_peered":"2024-04-22T07:04:12.787818+0000","last_unstale":"2024-04-22T07:04:15.055829+0000","last_undegraded":"2024-04-22T07:04:15.055829+0000","last_fullsized":"2024-04-22T07:04:15.055829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055442+0000","last_change":"2024-04-22T07:04:13.802013+0000","last_active":"2024-04-22T07:04:15.055442+0000","last_peered":"2024-04-22T07:04:15.055442+0000","last_clean":"2024-04-22T07:04:15.055442+0000","last_became_active":"2024-04-22T07:04:13.801404+0000","last_became_peered":"2024-04-22T07:04:13.801404+0000","last_unstale":"2024-04-22T07:04:15.055442+0000","last_undegraded":"2024-04-22T07:04:15.055442+0000","last_fullsized":"2024-04-22T07:04:15.055442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961382+0000","last_change":"2024-04-22T07:04:12.790981+0000","last_active":"2024-04-22T07:04:14.961382+0000","last_peered":"2024-04-22T07:04:14.961382+0000","last_clean":"2024-04-22T07:04:14.961382+0000","last_became_active":"2024-04-22T07:04:12.790592+0000","last_became_peered":"2024-04-22T07:04:12.790592+0000","last_unstale":"2024-04-22T07:04:14.961382+0000","last_undegraded":"2024-04-22T07:04:14.961382+0000","last_fullsized":"2024-04-22T07:04:14.961382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:53.115405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781878+0000","last_change":"2024-04-22T07:04:13.800619+0000","last_active":"2024-04-22T07:04:14.781878+0000","last_peered":"2024-04-22T07:04:14.781878+0000","last_clean":"2024-04-22T07:04:14.781878+0000","last_became_active":"2024-04-22T07:04:13.800279+0000","last_became_peered":"2024-04-22T07:04:13.800279+0000","last_unstale":"2024-04-22T07:04:14.781878+0000","last_undegraded":"2024-04-22T07:04:14.781878+0000","last_fullsized":"2024-04-22T07:04:14.781878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782949+0000","last_change":"2024-04-22T07:04:13.789060+0000","last_active":"2024-04-22T07:04:14.782949+0000","last_peered":"2024-04-22T07:04:14.782949+0000","last_clean":"2024-04-22T07:04:14.782949+0000","last_became_active":"2024-04-22T07:04:13.788787+0000","last_became_peered":"2024-04-22T07:04:13.788787+0000","last_unstale":"2024-04-22T07:04:14.782949+0000","last_undegraded":"2024-04-22T07:04:14.782949+0000","last_fullsized":"2024-04-22T07:04:14.782949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961028+0000","last_change":"2024-04-22T07:04:12.785077+0000","last_active":"2024-04-22T07:04:14.961028+0000","last_peered":"2024-04-22T07:04:14.961028+0000","last_clean":"2024-04-22T07:04:14.961028+0000","last_became_active":"2024-04-22T07:04:12.784804+0000","last_became_peered":"2024-04-22T07:04:12.784804+0000","last_unstale":"2024-04-22T07:04:14.961028+0000","last_undegraded":"2024-04-22T07:04:14.961028+0000","last_fullsized":"2024-04-22T07:04:14.961028+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783259+0000","last_change":"2024-04-22T07:04:13.789866+0000","last_active":"2024-04-22T07:04:14.783259+0000","last_peered":"2024-04-22T07:04:14.783259+0000","last_clean":"2024-04-22T07:04:14.783259+0000","last_became_active":"2024-04-22T07:04:13.789539+0000","last_became_peered":"2024-04-22T07:04:13.789539+0000","last_unstale":"2024-04-22T07:04:14.783259+0000","last_undegraded":"2024-04-22T07:04:14.783259+0000","last_fullsized":"2024-04-22T07:04:14.783259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961290+0000","last_change":"2024-04-22T07:04:12.786162+0000","last_active":"2024-04-22T07:04:14.961290+0000","last_peered":"2024-04-22T07:04:14.961290+0000","last_clean":"2024-04-22T07:04:14.961290+0000","last_became_active":"2024-04-22T07:04:12.785310+0000","last_became_peered":"2024-04-22T07:04:12.785310+0000","last_unstale":"2024-04-22T07:04:14.961290+0000","last_undegraded":"2024-04-22T07:04:14.961290+0000","last_fullsized":"2024-04-22T07:04:14.961290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:06.776205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056634+0000","last_change":"2024-04-22T07:04:13.796267+0000","last_active":"2024-04-22T07:04:15.056634+0000","last_peered":"2024-04-22T07:04:15.056634+0000","last_clean":"2024-04-22T07:04:15.056634+0000","last_became_active":"2024-04-22T07:04:13.795220+0000","last_became_peered":"2024-04-22T07:04:13.795220+0000","last_unstale":"2024-04-22T07:04:15.056634+0000","last_undegraded":"2024-04-22T07:04:15.056634+0000","last_fullsized":"2024-04-22T07:04:15.056634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055546+0000","last_change":"2024-04-22T07:04:12.785962+0000","last_active":"2024-04-22T07:04:15.055546+0000","last_peered":"2024-04-22T07:04:15.055546+0000","last_clean":"2024-04-22T07:04:15.055546+0000","last_became_active":"2024-04-22T07:04:12.785608+0000","last_became_peered":"2024-04-22T07:04:12.785608+0000","last_unstale":"2024-04-22T07:04:15.055546+0000","last_undegraded":"2024-04-22T07:04:15.055546+0000","last_fullsized":"2024-04-22T07:04:15.055546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055613+0000","last_change":"2024-04-22T07:04:13.801662+0000","last_active":"2024-04-22T07:04:15.055613+0000","last_peered":"2024-04-22T07:04:15.055613+0000","last_clean":"2024-04-22T07:04:15.055613+0000","last_became_active":"2024-04-22T07:04:13.798282+0000","last_became_peered":"2024-04-22T07:04:13.798282+0000","last_unstale":"2024-04-22T07:04:15.055613+0000","last_undegraded":"2024-04-22T07:04:15.055613+0000","last_fullsized":"2024-04-22T07:04:15.055613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056742+0000","last_change":"2024-04-22T07:04:12.791445+0000","last_active":"2024-04-22T07:04:15.056742+0000","last_peered":"2024-04-22T07:04:15.056742+0000","last_clean":"2024-04-22T07:04:15.056742+0000","last_became_active":"2024-04-22T07:04:12.791150+0000","last_became_peered":"2024-04-22T07:04:12.791150+0000","last_unstale":"2024-04-22T07:04:15.056742+0000","last_undegraded":"2024-04-22T07:04:15.056742+0000","last_fullsized":"2024-04-22T07:04:15.056742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":827392,"data_stored":671603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":221184,"data_stored":74935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":671603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:20.161 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-22T07:04:20.161 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-22T07:04:20.161 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-22T07:04:20.161 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-22T07:04:20.162 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-22T07:04:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.300+0000 7f90c2481640 1 Processor -- start 2024-04-22T07:04:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.300+0000 7f90c2481640 1 -- start start 2024-04-22T07:04:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90c2481640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc155050 0x7f90bc0cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90c2481640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f90bc15b7c0 0x7f90bc0ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90c2481640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90bc15e440 con 0x7f90bc0cd0a0 2024-04-22T07:04:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90c2481640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f90bc15f2a0 con 0x7f90bc15b7c0 2024-04-22T07:04:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90c2481640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f90bc15e720 con 0x7f90bc155050 2024-04-22T07:04:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90bb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f90bc15b7c0 0x7f90bc0ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90bbfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc155050 0x7f90bc0cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90bb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f90bc15b7c0 0x7f90bc0ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36672/0 (socket says 172.21.15.44:36672) 2024-04-22T07:04:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90bb7fe640 1 -- 172.21.15.44:0/2137002486 learned_addr learned my addr 172.21.15.44:0/2137002486 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90c0a1e640 1 --1- 172.21.15.44:0/2137002486 >> v1:172.21.15.70:6789/0 conn(0x7f90bc0cd0a0 0x7f90bc0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:51666/0 (socket says 172.21.15.44:51666) 2024-04-22T07:04:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90bb7fe640 1 -- 172.21.15.44:0/2137002486 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc155050 msgr2=0x7f90bc0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90bb7fe640 1 --2- 172.21.15.44:0/2137002486 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc155050 0x7f90bc0cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90bb7fe640 1 -- 172.21.15.44:0/2137002486 >> v1:172.21.15.70:6789/0 conn(0x7f90bc0cd0a0 legacy=0x7f90bc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90bb7fe640 1 -- 172.21.15.44:0/2137002486 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc15ea00 con 0x7f90bc15b7c0 2024-04-22T07:04:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.301+0000 7f90bbfff640 1 --2- 172.21.15.44:0/2137002486 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc155050 0x7f90bc0cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.302+0000 7f90bb7fe640 1 --2- 172.21.15.44:0/2137002486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90bc15b7c0 0x7f90bc0ccb60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009870 tx=0x7f90b002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb5f3abfe4ac7fa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.302+0000 7f90baffd640 1 -- 172.21.15.44:0/2137002486 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0034070 con 0x7f90bc15b7c0 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.302+0000 7f90baffd640 1 -- 172.21.15.44:0/2137002486 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b0030840 con 0x7f90bc15b7c0 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.302+0000 7f90baffd640 1 -- 172.21.15.44:0/2137002486 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0030c10 con 0x7f90bc15b7c0 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.303+0000 7f90c2481640 1 -- 172.21.15.44:0/2137002486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90bc15b7c0 msgr2=0x7f90bc0ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.303+0000 7f90c2481640 1 --2- 172.21.15.44:0/2137002486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90bc15b7c0 0x7f90bc0ccb60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009870 tx=0x7f90b002b980 comp rx=0 tx=0).stop 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.303+0000 7f90c2481640 1 -- 172.21.15.44:0/2137002486 shutdown_connections 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.303+0000 7f90c2481640 1 --2- 172.21.15.44:0/2137002486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90bc15b7c0 0x7f90bc0ccb60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.303+0000 7f90c2481640 1 --2- 172.21.15.44:0/2137002486 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc155050 0x7f90bc0cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.303+0000 7f90c2481640 1 -- 172.21.15.44:0/2137002486 >> 172.21.15.44:0/2137002486 conn(0x7f90bc06b970 msgr2=0x7f90bc06bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.303+0000 7f90c2481640 1 -- 172.21.15.44:0/2137002486 shutdown_connections 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.303+0000 7f90c2481640 1 -- 172.21.15.44:0/2137002486 wait complete. 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.304+0000 7f90c2481640 1 Processor -- start 2024-04-22T07:04:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.304+0000 7f90c2481640 1 -- start start 2024-04-22T07:04:20.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c2481640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc0cd0a0 0x7f90bc159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c2481640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90bc155050 0x7f90bc155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c2481640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15b7c0 0x7f90bc1e4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c2481640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc155430 con 0x7f90bc15b7c0 2024-04-22T07:04:20.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c2481640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc15efc0 con 0x7f90bc155050 2024-04-22T07:04:20.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c2481640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f90bc15bc20 con 0x7f90bc0cd0a0 2024-04-22T07:04:20.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c0a1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15b7c0 0x7f90bc1e4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c0a1e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15b7c0 0x7f90bc1e4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37866/0 (socket says 172.21.15.44:37866) 2024-04-22T07:04:20.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c0a1e640 1 -- 172.21.15.44:0/3027015137 learned_addr learned my addr 172.21.15.44:0/3027015137 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:20.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90bbfff640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc0cd0a0 0x7f90bc159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90bb7fe640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90bc155050 0x7f90bc155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c0a1e640 1 -- 172.21.15.44:0/3027015137 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc0cd0a0 msgr2=0x7f90bc159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c0a1e640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc0cd0a0 0x7f90bc159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c0a1e640 1 -- 172.21.15.44:0/3027015137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90bc155050 msgr2=0x7f90bc155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c0a1e640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90bc155050 0x7f90bc155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c0a1e640 1 -- 172.21.15.44:0/3027015137 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc158b40 con 0x7f90bc15b7c0 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90bb7fe640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90bc155050 0x7f90bc155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c0a1e640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15b7c0 0x7f90bc1e4130 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c7b0 tx=0x7f90b400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90b8ff9640 1 -- 172.21.15.44:0/3027015137 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4017070 con 0x7f90bc15b7c0 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90b8ff9640 1 -- 172.21.15.44:0/3027015137 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b400a640 con 0x7f90bc15b7c0 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c2481640 1 -- 172.21.15.44:0/3027015137 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc158de0 con 0x7f90bc15b7c0 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90b8ff9640 1 -- 172.21.15.44:0/3027015137 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b400a9f0 con 0x7f90bc15b7c0 2024-04-22T07:04:20.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.305+0000 7f90c2481640 1 -- 172.21.15.44:0/3027015137 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc1593a0 con 0x7f90bc15b7c0 2024-04-22T07:04:20.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.307+0000 7f90c2481640 1 -- 172.21.15.44:0/3027015137 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90bc0ca950 con 0x7f90bc15b7c0 2024-04-22T07:04:20.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.307+0000 7f90b8ff9640 1 -- 172.21.15.44:0/3027015137 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90b401a070 con 0x7f90bc15b7c0 2024-04-22T07:04:20.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.307+0000 7f90b8ff9640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f90a806c920 0x7f90a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.307+0000 7f90b8ff9640 1 -- 172.21.15.44:0/3027015137 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f90b408b050 con 0x7f90bc15b7c0 2024-04-22T07:04:20.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.307+0000 7f90bbfff640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f90a806c920 0x7f90a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.308 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.308+0000 7f90bbfff640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f90a806c920 0x7f90a806ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009560 tx=0x7f90ac009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:20.310 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.312+0000 7f90b8ff9640 1 -- 172.21.15.44:0/3027015137 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90b4058e30 con 0x7f90bc15b7c0 2024-04-22T07:04:20.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.549+0000 7f90c2481640 1 -- 172.21.15.44:0/3027015137 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f90bc153d20 con 0x7f90bc15b7c0 2024-04-22T07:04:20.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.552+0000 7f90b8ff9640 1 -- 172.21.15.44:0/3027015137 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174136 (secure 0 0 0) 0x7f90b405d130 con 0x7f90bc15b7c0 2024-04-22T07:04:20.553 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:20.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 -- 172.21.15.44:0/3027015137 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f90a806c920 msgr2=0x7f90a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f90a806c920 0x7f90a806ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009560 tx=0x7f90ac009190 comp rx=0 tx=0).stop 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 -- 172.21.15.44:0/3027015137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15b7c0 msgr2=0x7f90bc1e4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15b7c0 0x7f90bc1e4130 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f90b400c7b0 tx=0x7f90b400cc70 comp rx=0 tx=0).stop 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 -- 172.21.15.44:0/3027015137 shutdown_connections 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f90a806c920 0x7f90a806ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f90bc15b7c0 0x7f90bc1e4130 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90bc155050 0x7f90bc155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 --2- 172.21.15.44:0/3027015137 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f90bc0cd0a0 0x7f90bc159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 -- 172.21.15.44:0/3027015137 >> 172.21.15.44:0/3027015137 conn(0x7f90bc06b970 msgr2=0x7f90bc14add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 -- 172.21.15.44:0/3027015137 shutdown_connections 2024-04-22T07:04:20.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.555+0000 7f90c2481640 1 -- 172.21.15.44:0/3027015137 wait complete. 2024-04-22T07:04:20.564 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6832","nonce":1889438723},{"type":"v1","addr":"172.21.15.44:6833","nonce":1889438723}]},"active_addr":"172.21.15.44:6833/1889438723","active_change":"2024-04-22T07:03:53.701359+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4115,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":3250718477}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":1606293146}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":401836767}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.44:0","nonce":2364620137}]}]} 2024-04-22T07:04:20.568 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-22T07:04:20.568 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-22T07:04:20.568 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:20.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3ebdb4a640 1 Processor -- start 2024-04-22T07:04:20.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3ebdb4a640 1 -- start start 2024-04-22T07:04:20.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3ebdb4a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3eb80c4430 0x7f3eb80c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3ebdb4a640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb80c3970 con 0x7f3eb81564a0 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3ebdb4a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb80c9f60 con 0x7f3eb81560c0 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3ebdb4a640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f3eb8158e00 con 0x7f3eb80c4430 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3eb6ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3eb80c4430 0x7f3eb80c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3eb6ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3eb80c4430 0x7f3eb80c3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37878/0 (socket says 172.21.15.44:37878) 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3eb7fff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f3eb81564a0 0x7f3eb8159290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38026/0 (socket says 172.21.15.44:38026) 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3eb6ffd640 1 -- 172.21.15.44:0/4266214182 learned_addr learned my addr 172.21.15.44:0/4266214182 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3eb6ffd640 1 -- 172.21.15.44:0/4266214182 >> v1:172.21.15.70:6789/0 conn(0x7f3eb81560c0 legacy=0x7f3eb80c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3eb6ffd640 1 -- 172.21.15.44:0/4266214182 >> v1:172.21.15.44:6789/0 conn(0x7f3eb81564a0 legacy=0x7f3eb8159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3eb6ffd640 1 -- 172.21.15.44:0/4266214182 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb80ca240 con 0x7f3eb80c4430 2024-04-22T07:04:20.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.671+0000 7f3eb67fc640 1 -- 172.21.15.44:0/4266214182 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3014054836 0 0) 0x7f3eb80c3970 con 0x7f3eb81564a0 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3eb6ffd640 1 --2- 172.21.15.44:0/4266214182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb80c4430 0x7f3eb80c3380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009870 tx=0x7f3eac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24d499797029060 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3eb67fc640 1 -- 172.21.15.44:0/4266214182 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac034070 con 0x7f3eb80c4430 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3eb67fc640 1 -- 172.21.15.44:0/4266214182 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eac030860 con 0x7f3eb80c4430 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3eb67fc640 1 -- 172.21.15.44:0/4266214182 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac030c30 con 0x7f3eb80c4430 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/4266214182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb80c4430 msgr2=0x7f3eb80c3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3ebdb4a640 1 --2- 172.21.15.44:0/4266214182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb80c4430 0x7f3eb80c3380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009870 tx=0x7f3eac02b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/4266214182 shutdown_connections 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3ebdb4a640 1 --2- 172.21.15.44:0/4266214182 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb80c4430 0x7f3eb80c3380 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/4266214182 >> 172.21.15.44:0/4266214182 conn(0x7f3eb80713b0 msgr2=0x7f3eb80717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/4266214182 shutdown_connections 2024-04-22T07:04:20.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.672+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/4266214182 wait complete. 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.674+0000 7f3ebdb4a640 1 Processor -- start 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.674+0000 7f3ebdb4a640 1 -- start start 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.674+0000 7f3ebdb4a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb81560c0 0x7f3eb80ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.674+0000 7f3ebdb4a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb81564a0 0x7f3eb806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.674+0000 7f3ebdb4a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806d070 0x7f3eb80b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.674+0000 7f3ebdb4a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb80caae0 con 0x7f3eb806d070 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.674+0000 7f3ebdb4a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb80ca800 con 0x7f3eb81564a0 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.674+0000 7f3ebdb4a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb80cf080 con 0x7f3eb81560c0 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806d070 0x7f3eb80b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806d070 0x7f3eb80b2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37884/0 (socket says 172.21.15.44:37884) 2024-04-22T07:04:20.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb7fff640 1 -- 172.21.15.44:0/2644565090 learned_addr learned my addr 172.21.15.44:0/2644565090 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb77fe640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb81560c0 0x7f3eb80ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb6ffd640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb81564a0 0x7f3eb806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb7fff640 1 -- 172.21.15.44:0/2644565090 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb81560c0 msgr2=0x7f3eb80ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb7fff640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb81560c0 0x7f3eb80ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb7fff640 1 -- 172.21.15.44:0/2644565090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb81564a0 msgr2=0x7f3eb806cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb7fff640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb81564a0 0x7f3eb806cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb7fff640 1 -- 172.21.15.44:0/2644565090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb80aa5c0 con 0x7f3eb806d070 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb77fe640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb81560c0 0x7f3eb80ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3eb7fff640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806d070 0x7f3eb80b2b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000c790 tx=0x7f3eb000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3e9ffff640 1 -- 172.21.15.44:0/2644565090 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb000a640 con 0x7f3eb806d070 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3e9ffff640 1 -- 172.21.15.44:0/2644565090 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb000a800 con 0x7f3eb806d070 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/2644565090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb81f1340 con 0x7f3eb806d070 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3e9ffff640 1 -- 172.21.15.44:0/2644565090 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb000abf0 con 0x7f3eb806d070 2024-04-22T07:04:20.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.675+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/2644565090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb806c330 con 0x7f3eb806d070 2024-04-22T07:04:20.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.676+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/2644565090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb8069ea0 con 0x7f3eb806d070 2024-04-22T07:04:20.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.677+0000 7f3e9ffff640 1 -- 172.21.15.44:0/2644565090 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3eb0007070 con 0x7f3eb806d070 2024-04-22T07:04:20.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.677+0000 7f3e9ffff640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3ea406c8f0 0x7f3ea406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.677+0000 7f3e9ffff640 1 -- 172.21.15.44:0/2644565090 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3eb008bae0 con 0x7f3eb806d070 2024-04-22T07:04:20.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.677+0000 7f3eb77fe640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3ea406c8f0 0x7f3ea406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.678+0000 7f3eb77fe640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3ea406c8f0 0x7f3ea406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8003eb0 tx=0x7f3ea8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:20.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.681+0000 7f3e9ffff640 1 -- 172.21.15.44:0/2644565090 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3eb0059f10 con 0x7f3eb806d070 2024-04-22T07:04:20.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.871+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/2644565090 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3eb80c52b0 con 0x7f3eb806d070 2024-04-22T07:04:20.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.872+0000 7f3e9ffff640 1 -- 172.21.15.44:0/2644565090 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16557 (secure 0 0 0) 0x7f3eb0059810 con 0x7f3eb806d070 2024-04-22T07:04:20.871 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:20.871 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":23,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:14.778262+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:20.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/2644565090 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3ea406c8f0 msgr2=0x7f3ea406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3ea406c8f0 0x7f3ea406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8003eb0 tx=0x7f3ea8009040 comp rx=0 tx=0).stop 2024-04-22T07:04:20.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/2644565090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806d070 msgr2=0x7f3eb80b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806d070 0x7f3eb80b2b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000c790 tx=0x7f3eb000cc50 comp rx=0 tx=0).stop 2024-04-22T07:04:20.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/2644565090 shutdown_connections 2024-04-22T07:04:20.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3ea406c8f0 0x7f3ea406edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3eb806d070 0x7f3eb80b2b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3eb81564a0 0x7f3eb806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 --2- 172.21.15.44:0/2644565090 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3eb81560c0 0x7f3eb80ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/2644565090 >> 172.21.15.44:0/2644565090 conn(0x7f3eb80713b0 msgr2=0x7f3eb8071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:20.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/2644565090 shutdown_connections 2024-04-22T07:04:20.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.875+0000 7f3ebdb4a640 1 -- 172.21.15.44:0/2644565090 wait complete. 2024-04-22T07:04:20.884 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-22T07:04:20.884 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:20.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.985+0000 7f13ca7b6640 1 Processor -- start 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.985+0000 7f13ca7b6640 1 -- start start 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13ca7b6640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f13c40c6ca0 0x7f13c40708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13ca7b6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40c7080 0x7f13c4070e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13ca7b6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13c406b970 0x7f13c4152780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13ca7b6640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f13c40703a0 con 0x7f13c40c6ca0 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13ca7b6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f13c40c3e10 con 0x7f13c406b970 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13ca7b6640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f13c40c40d0 con 0x7f13c40c7080 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13c3fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f13c40c6ca0 0x7f13c40708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13c3fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f13c40c6ca0 0x7f13c40708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37888/0 (socket says 172.21.15.44:37888) 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13c3fff640 1 -- 172.21.15.44:0/3439660872 learned_addr learned my addr 172.21.15.44:0/3439660872 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13c37fe640 1 --2- 172.21.15.44:0/3439660872 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40c7080 0x7f13c4070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.986+0000 7f13c8d53640 1 --2- 172.21.15.44:0/3439660872 >> v2:172.21.15.70:3300/0 conn(0x7f13c406b970 0x7f13c4152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.987+0000 7f13c3fff640 1 -- 172.21.15.44:0/3439660872 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40c7080 msgr2=0x7f13c4070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.987+0000 7f13c3fff640 1 --2- 172.21.15.44:0/3439660872 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40c7080 0x7f13c4070e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.987+0000 7f13c3fff640 1 -- 172.21.15.44:0/3439660872 >> v2:172.21.15.70:3300/0 conn(0x7f13c406b970 msgr2=0x7f13c4152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.987+0000 7f13c3fff640 1 --2- 172.21.15.44:0/3439660872 >> v2:172.21.15.70:3300/0 conn(0x7f13c406b970 0x7f13c4152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.987+0000 7f13c3fff640 1 -- 172.21.15.44:0/3439660872 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c40cde60 con 0x7f13c40c6ca0 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.987+0000 7f13c8d53640 1 --2- 172.21.15.44:0/3439660872 >> v2:172.21.15.70:3300/0 conn(0x7f13c406b970 0x7f13c4152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.987+0000 7f13c3fff640 1 --2- 172.21.15.44:0/3439660872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40c6ca0 0x7f13c40708c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f13c406e620 tx=0x7f13b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e025ca46255a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13c2ffd640 1 -- 172.21.15.44:0/3439660872 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4034070 con 0x7f13c40c6ca0 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13c2ffd640 1 -- 172.21.15.44:0/3439660872 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b4030860 con 0x7f13c40c6ca0 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13c2ffd640 1 -- 172.21.15.44:0/3439660872 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4030c30 con 0x7f13c40c6ca0 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13ca7b6640 1 -- 172.21.15.44:0/3439660872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40c6ca0 msgr2=0x7f13c40708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13ca7b6640 1 --2- 172.21.15.44:0/3439660872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40c6ca0 0x7f13c40708c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f13c406e620 tx=0x7f13b402b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13ca7b6640 1 -- 172.21.15.44:0/3439660872 shutdown_connections 2024-04-22T07:04:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13ca7b6640 1 --2- 172.21.15.44:0/3439660872 >> v2:172.21.15.70:3300/0 conn(0x7f13c406b970 0x7f13c4152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13ca7b6640 1 --2- 172.21.15.44:0/3439660872 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40c7080 0x7f13c4070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13ca7b6640 1 --2- 172.21.15.44:0/3439660872 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40c6ca0 0x7f13c40708c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13ca7b6640 1 -- 172.21.15.44:0/3439660872 >> 172.21.15.44:0/3439660872 conn(0x7f13c40727b0 msgr2=0x7f13c4072bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13ca7b6640 1 -- 172.21.15.44:0/3439660872 shutdown_connections 2024-04-22T07:04:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.988+0000 7f13ca7b6640 1 -- 172.21.15.44:0/3439660872 wait complete. 2024-04-22T07:04:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.989+0000 7f13ca7b6640 1 Processor -- start 2024-04-22T07:04:20.991 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13ca7b6640 1 -- start start 2024-04-22T07:04:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13ca7b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c406b970 0x7f13c4158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13ca7b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40c6ca0 0x7f13c4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13ca7b6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40c7080 0x7f13c4145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13ca7b6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13c40c80a0 con 0x7f13c406b970 2024-04-22T07:04:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13ca7b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f13c40c7dc0 con 0x7f13c40c6ca0 2024-04-22T07:04:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13ca7b6640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f13c40c8380 con 0x7f13c40c7080 2024-04-22T07:04:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13c3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c406b970 0x7f13c4158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13c3fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c406b970 0x7f13c4158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37896/0 (socket says 172.21.15.44:37896) 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13c3fff640 1 -- 172.21.15.44:0/4118869988 learned_addr learned my addr 172.21.15.44:0/4118869988 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13c37fe640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40c6ca0 0x7f13c4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.990+0000 7f13c8d53640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40c7080 0x7f13c4145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13c3fff640 1 -- 172.21.15.44:0/4118869988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40c7080 msgr2=0x7f13c4145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13c3fff640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40c7080 0x7f13c4145b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13c3fff640 1 -- 172.21.15.44:0/4118869988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40c6ca0 msgr2=0x7f13c4158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13c3fff640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40c6ca0 0x7f13c4158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13c3fff640 1 -- 172.21.15.44:0/4118869988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c41ef190 con 0x7f13c406b970 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13c3fff640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c406b970 0x7f13c4158a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13b4000c00 tx=0x7f13b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13c0ff9640 1 -- 172.21.15.44:0/4118869988 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4048070 con 0x7f13c406b970 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13c0ff9640 1 -- 172.21.15.44:0/4118869988 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b4030660 con 0x7f13c406b970 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13c0ff9640 1 -- 172.21.15.44:0/4118869988 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4034040 con 0x7f13c406b970 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13ca7b6640 1 -- 172.21.15.44:0/4118869988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c41498d0 con 0x7f13c406b970 2024-04-22T07:04:20.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.991+0000 7f13ca7b6640 1 -- 172.21.15.44:0/4118869988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c4158560 con 0x7f13c406b970 2024-04-22T07:04:20.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.992+0000 7f13ca7b6640 1 -- 172.21.15.44:0/4118869988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c404fdc0 con 0x7f13c406b970 2024-04-22T07:04:20.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.992+0000 7f13c0ff9640 1 -- 172.21.15.44:0/4118869988 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13b404b070 con 0x7f13c406b970 2024-04-22T07:04:20.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.993+0000 7f13c0ff9640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c8f0 0x7f13b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:20.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.993+0000 7f13c0ff9640 1 -- 172.21.15.44:0/4118869988 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f13b40adc30 con 0x7f13c406b970 2024-04-22T07:04:20.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.993+0000 7f13c37fe640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c8f0 0x7f13b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:20.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.993+0000 7f13c37fe640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c8f0 0x7f13b006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f13c414ed60 tx=0x7f13b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:20.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:20.997+0000 7f13c0ff9640 1 -- 172.21.15.44:0/4118869988 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13b407b980 con 0x7f13c406b970 2024-04-22T07:04:21.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.183+0000 7f13ca7b6640 1 -- 172.21.15.44:0/4118869988 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13c414fee0 con 0x7f13c406b970 2024-04-22T07:04:21.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.184+0000 7f13c0ff9640 1 -- 172.21.15.44:0/4118869988 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16557 (secure 0 0 0) 0x7f13b407fc80 con 0x7f13c406b970 2024-04-22T07:04:21.183 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:21.183 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":23,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:14.778262+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:21.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 -- 172.21.15.44:0/4118869988 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c8f0 msgr2=0x7f13b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c8f0 0x7f13b006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f13c414ed60 tx=0x7f13b8008040 comp rx=0 tx=0).stop 2024-04-22T07:04:21.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 -- 172.21.15.44:0/4118869988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c406b970 msgr2=0x7f13c4158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c406b970 0x7f13c4158a80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13b4000c00 tx=0x7f13b4004730 comp rx=0 tx=0).stop 2024-04-22T07:04:21.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 -- 172.21.15.44:0/4118869988 shutdown_connections 2024-04-22T07:04:21.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c8f0 0x7f13b006edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40c7080 0x7f13c4145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40c6ca0 0x7f13c4158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 --2- 172.21.15.44:0/4118869988 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c406b970 0x7f13c4158a80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 -- 172.21.15.44:0/4118869988 >> 172.21.15.44:0/4118869988 conn(0x7f13c40727b0 msgr2=0x7f13c40c9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 -- 172.21.15.44:0/4118869988 shutdown_connections 2024-04-22T07:04:21.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.186+0000 7f13ca7b6640 1 -- 172.21.15.44:0/4118869988 wait complete. 2024-04-22T07:04:21.196 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-22T07:04:21.197 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-22T07:04:21.197 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-22T07:04:21.197 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-22T07:04:21.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-22T07:04:21.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-22T07:04:21.199 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-22T07:04:21.199 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-22T07:04:21.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.375+0000 7f3da3063640 1 Processor -- start 2024-04-22T07:04:21.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.375+0000 7f13dccb3640 1 Processor -- start 2024-04-22T07:04:21.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7f13dccb3640 1 -- start start 2024-04-22T07:04:21.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7fbb19e55640 1 Processor -- start 2024-04-22T07:04:21.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.375+0000 7f3da3063640 1 -- start start 2024-04-22T07:04:21.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7fbb19e55640 1 -- start start 2024-04-22T07:04:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.375+0000 7f13acc74640 1 Processor -- start 2024-04-22T07:04:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.375+0000 7f13acc74640 1 -- start start 2024-04-22T07:04:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7f33c15fa640 1 Processor -- start 2024-04-22T07:04:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7f33c15fa640 1 -- start start 2024-04-22T07:04:21.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7f663bfff640 1 Processor -- start 2024-04-22T07:04:21.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7f3da3063640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d940d5dc0 0x7f3d9404d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7f3da3063640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d940dc390 0x7f3d94046230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f663bfff640 1 -- start start 2024-04-22T07:04:21.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7f3da3063640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d94047110 con 0x7f3d94047e60 2024-04-22T07:04:21.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7f3da3063640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f3d9404cc20 con 0x7f3d940d5dc0 2024-04-22T07:04:21.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13acc74640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13a004ba60 0x7f13a004be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13acc74640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f13a00e3090 0x7f13a00e34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.376+0000 7f3da3063640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3d940dc840 con 0x7f3d940dc390 2024-04-22T07:04:21.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13acc74640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a0046b70 con 0x7f13a00dd3c0 2024-04-22T07:04:21.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13acc74640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f13a0045450 con 0x7f13a00e3090 2024-04-22T07:04:21.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13acc74640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f13a0045fd0 con 0x7f13a004ba60 2024-04-22T07:04:21.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13a6d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f13a00e3090 0x7f13a00e34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13a6d9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f13a00e3090 0x7f13a00e34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37906/0 (socket says 172.21.15.44:37906) 2024-04-22T07:04:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13dccb3640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d806c4f0 0x7f13d815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13dccb3640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d80c65b0 con 0x7f13d8154ba0 2024-04-22T07:04:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13dccb3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d80c5510 con 0x7f13d80d0730 2024-04-22T07:04:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13dccb3640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f13d80c52c0 con 0x7f13d806c4f0 2024-04-22T07:04:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13d1e9b640 1 Processor -- start 2024-04-22T07:04:21.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f3da1600640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d940dc390 0x7f3d94046230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7fbb19e55640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbb141592d0 0x7fbb140716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7fbb19e55640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbb14163160 0x7fbb14072460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7fbb19e55640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb1415f370 con 0x7fbb140729a0 2024-04-22T07:04:21.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7fbb19e55640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fbb14071d70 con 0x7fbb141592d0 2024-04-22T07:04:21.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7fbb19e55640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbb1415ff40 con 0x7fbb14163160 2024-04-22T07:04:21.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f3da0dff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d940d5dc0 0x7f3d9404d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13d1e9b640 1 -- start start 2024-04-22T07:04:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13a6d9d640 1 -- 172.21.15.44:0/4180568215 learned_addr learned my addr 172.21.15.44:0/4180568215 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f3da0dff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d940d5dc0 0x7f3d9404d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36702/0 (socket says 172.21.15.44:36702) 2024-04-22T07:04:21.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f3da0dff640 1 -- 172.21.15.44:0/2586965928 learned_addr learned my addr 172.21.15.44:0/2586965928 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13d77fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f13d8154ba0 0x7f13d815bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38042/0 (socket says 172.21.15.44:38042) 2024-04-22T07:04:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13d77fe640 1 -- 172.21.15.44:0/147561741 learned_addr learned my addr 172.21.15.44:0/147561741 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13a5d9b640 1 --2- 172.21.15.44:0/4180568215 >> v2:172.21.15.70:3300/0 conn(0x7f13a004ba60 0x7f13a004be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7fbb12ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbb14163160 0x7fbb14072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7fbb13fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbb140729a0 0x7fbb140cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:51696/0 (socket says 172.21.15.44:51696) 2024-04-22T07:04:21.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f13d6ffd640 1 --2- 172.21.15.44:0/147561741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d806c4f0 0x7f13d815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7fbb13fff640 1 -- 172.21.15.44:0/913267306 learned_addr learned my addr 172.21.15.44:0/913267306 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:21.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f663bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66340d6f20 0x7f663404c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f13d67fc640 1 -- 172.21.15.44:0/147561741 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1691162826 0 0) 0x7f13d80c5510 con 0x7f13d80d0730 2024-04-22T07:04:21.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f13d67fc640 1 -- 172.21.15.44:0/147561741 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13c0003520 con 0x7f13d80d0730 2024-04-22T07:04:21.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f663bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6634004a30 0x7f663404bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f663bfff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66340dcaa0 con 0x7f6634033910 2024-04-22T07:04:21.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f663bfff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f66340dcd60 con 0x7f6634004a30 2024-04-22T07:04:21.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f663bfff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f66340dd040 con 0x7f66340d6f20 2024-04-22T07:04:21.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7fbb137fe640 1 --2- 172.21.15.44:0/913267306 >> v2:172.21.15.44:3300/0 conn(0x7fbb141592d0 0x7fbb140716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f663a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6634004a30 0x7f663404bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f663b7fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f6634033910 0x7f66340e0240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38056/0 (socket says 172.21.15.44:38056) 2024-04-22T07:04:21.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f663a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6634004a30 0x7f663404bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36726/0 (socket says 172.21.15.44:36726) 2024-04-22T07:04:21.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f663b7fe640 1 -- 172.21.15.44:0/2318127264 learned_addr learned my addr 172.21.15.44:0/2318127264 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f3da0dff640 1 -- 172.21.15.44:0/2586965928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d940dc390 msgr2=0x7f3d94046230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f3da0dff640 1 --2- 172.21.15.44:0/2586965928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d940dc390 0x7f3d94046230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f3da0dff640 1 -- 172.21.15.44:0/2586965928 >> v1:172.21.15.70:6789/0 conn(0x7f3d94047e60 legacy=0x7f3d94007c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f3da0dff640 1 -- 172.21.15.44:0/2586965928 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9404c280 con 0x7f3d940d5dc0 2024-04-22T07:04:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f13a5d9b640 1 -- 172.21.15.44:0/4180568215 >> v2:172.21.15.44:3300/0 conn(0x7f13a00e3090 msgr2=0x7f13a00e34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f13a5d9b640 1 --2- 172.21.15.44:0/4180568215 >> v2:172.21.15.44:3300/0 conn(0x7f13a00e3090 0x7f13a00e34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f13a5d9b640 1 -- 172.21.15.44:0/4180568215 >> v1:172.21.15.44:6789/0 conn(0x7f13a00dd3c0 legacy=0x7f13a0046460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f13a5d9b640 1 -- 172.21.15.44:0/4180568215 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a00dd7a0 con 0x7f13a004ba60 2024-04-22T07:04:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f13d1e9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13c40d9040 0x7f13c40061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7fbb12ffd640 1 -- 172.21.15.44:0/913267306 >> v2:172.21.15.44:3300/0 conn(0x7fbb141592d0 msgr2=0x7fbb140716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7fbb12ffd640 1 --2- 172.21.15.44:0/913267306 >> v2:172.21.15.44:3300/0 conn(0x7fbb141592d0 0x7fbb140716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7fbb12ffd640 1 -- 172.21.15.44:0/913267306 >> v1:172.21.15.70:6789/0 conn(0x7fbb140729a0 legacy=0x7fbb140cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7fbb12ffd640 1 -- 172.21.15.44:0/913267306 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1415f610 con 0x7fbb14163160 2024-04-22T07:04:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f13d1e9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f13c40508e0 0x7f13c40e2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.377+0000 7f537d481640 1 Processor -- start 2024-04-22T07:04:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f537d481640 1 -- start start 2024-04-22T07:04:21.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f537d481640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53781592c0 0x7f5378162580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f537d481640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f53780c3480 0x7f537815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f537d481640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f53780c9c80 0x7f53780c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f13d6ffd640 1 -- 172.21.15.44:0/147561741 >> v1:172.21.15.70:6789/0 conn(0x7f13d80d0730 legacy=0x7f13d81653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f13d6ffd640 1 -- 172.21.15.44:0/147561741 >> v1:172.21.15.44:6789/0 conn(0x7f13d8154ba0 legacy=0x7f13d815bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.378+0000 7f13d6ffd640 1 -- 172.21.15.44:0/147561741 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d80c57f0 con 0x7f13d806c4f0 2024-04-22T07:04:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f537d481640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f53781634d0 con 0x7f53780c3480 2024-04-22T07:04:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f537d481640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f537815f590 con 0x7f53780c9c80 2024-04-22T07:04:21.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f537d481640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f53780ca950 con 0x7f53781592c0 2024-04-22T07:04:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f13d0e99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13c40d9040 0x7f13c40061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.391 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f3da0dff640 1 --2- 172.21.15.44:0/2586965928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d940d5dc0 0x7f3d9404d7b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c065320 tx=0x7f3d9c0a7b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fee06366ffbe4859 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f13d0e99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13c40d9040 0x7f13c40061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36742/0 (socket says 172.21.15.44:36742) 2024-04-22T07:04:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f13a5d9b640 1 --2- 172.21.15.44:0/4180568215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13a004ba60 0x7f13a004be80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f13a802db50 tx=0x7f13a80a7a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd317734405528b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f663a7fc640 1 -- 172.21.15.44:0/2318127264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66340d6f20 msgr2=0x7f663404c8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:04:21.392 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f13d6ffd640 1 --2- 172.21.15.44:0/147561741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d806c4f0 0x7f13d815b4b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009dc0 tx=0x7f13d002bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c46f78bdd3053942 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f663a7fc640 1 --2- 172.21.15.44:0/2318127264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66340d6f20 0x7f663404c8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f663a7fc640 1 -- 172.21.15.44:0/2318127264 >> v1:172.21.15.44:6789/0 conn(0x7f6634033910 legacy=0x7f66340e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f663a7fc640 1 -- 172.21.15.44:0/2318127264 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66340dd320 con 0x7f6634004a30 2024-04-22T07:04:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f13d0e99640 1 -- 172.21.15.44:0/255246690 learned_addr learned my addr 172.21.15.44:0/255246690 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f3d9b7fe640 1 -- 172.21.15.44:0/2586965928 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0a99b0 con 0x7f3d940d5dc0 2024-04-22T07:04:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f13d67fc640 1 -- 172.21.15.44:0/147561741 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0039070 con 0x7f13d806c4f0 2024-04-22T07:04:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f3d9b7fe640 1 -- 172.21.15.44:0/2586965928 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9c0a9b50 con 0x7f3d940d5dc0 2024-04-22T07:04:21.394 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f13d67fc640 1 -- 172.21.15.44:0/147561741 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d002ccc0 con 0x7f13d806c4f0 2024-04-22T07:04:21.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f33c15fa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f33b40d5200 0x7f33b404b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f53767fc640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f53780c3480 0x7f537815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.395 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f13d67fc640 1 -- 172.21.15.44:0/147561741 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0039070 con 0x7f13d806c4f0 2024-04-22T07:04:21.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f13d1e9b640 1 -- 172.21.15.44:0/255246690 --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c4006860 con 0x7f13c40e2850 2024-04-22T07:04:21.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f13d1e9b640 1 -- 172.21.15.44:0/255246690 --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f13c4006b40 con 0x7f13c40508e0 2024-04-22T07:04:21.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f3d9b7fe640 1 -- 172.21.15.44:0/2586965928 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0b1420 con 0x7f3d940d5dc0 2024-04-22T07:04:21.396 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f13d1e9b640 1 -- 172.21.15.44:0/255246690 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f13c4006e20 con 0x7f13c40d9040 2024-04-22T07:04:21.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f5376ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53781592c0 0x7f5378162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f13a559a640 1 -- 172.21.15.44:0/4180568215 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a80ab920 con 0x7f13a004ba60 2024-04-22T07:04:21.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f13a559a640 1 -- 172.21.15.44:0/4180568215 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a80abb00 con 0x7f13a004ba60 2024-04-22T07:04:21.397 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f13a559a640 1 -- 172.21.15.44:0/4180568215 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a80b3400 con 0x7f13a004ba60 2024-04-22T07:04:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f53767fc640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f53780c3480 0x7f537815fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37916/0 (socket says 172.21.15.44:37916) 2024-04-22T07:04:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f53767fc640 1 -- 172.21.15.44:0/926470264 learned_addr learned my addr 172.21.15.44:0/926470264 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f53777fe640 1 --2- 172.21.15.44:0/926470264 >> v2:172.21.15.70:3300/0 conn(0x7f53780c9c80 0x7f53780c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.398 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f663a7fc640 1 --2- 172.21.15.44:0/2318127264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6634004a30 0x7f663404bd80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f662c0097a0 tx=0x7f662c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6f115629fa554ff server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f33c15fa640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33b400e2a0 0x7f33b4045330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f33c15fa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b4045ed0 con 0x7f33b404b090 2024-04-22T07:04:21.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f33c15fa640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f33b4046f30 con 0x7f33b40d5200 2024-04-22T07:04:21.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f33c15fa640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f33b4046170 con 0x7f33b400e2a0 2024-04-22T07:04:21.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7fbb12ffd640 1 --2- 172.21.15.44:0/913267306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb14163160 0x7fbb14072460 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb14070c30 tx=0x7fbb0802b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=487b59499259bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f53767fc640 1 -- 172.21.15.44:0/926470264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53781592c0 msgr2=0x7f5378162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.399 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f53767fc640 1 --2- 172.21.15.44:0/926470264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53781592c0 0x7f5378162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f53767fc640 1 -- 172.21.15.44:0/926470264 >> v2:172.21.15.70:3300/0 conn(0x7f53780c9c80 msgr2=0x7f53780c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f53767fc640 1 --2- 172.21.15.44:0/926470264 >> v2:172.21.15.70:3300/0 conn(0x7f53780c9c80 0x7f53780c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.379+0000 7f53767fc640 1 -- 172.21.15.44:0/926470264 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537815ff60 con 0x7f53780c3480 2024-04-22T07:04:21.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f3da3063640 1 -- 172.21.15.44:0/2586965928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d940d5dc0 msgr2=0x7f3d9404d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f3da3063640 1 --2- 172.21.15.44:0/2586965928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d940d5dc0 0x7f3d9404d7b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c065320 tx=0x7f3d9c0a7b30 comp rx=0 tx=0).stop 2024-04-22T07:04:21.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f3da3063640 1 -- 172.21.15.44:0/2586965928 shutdown_connections 2024-04-22T07:04:21.400 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f33bbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f33b40d5200 0x7f33b404b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f33bbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f33b40d5200 0x7f33b404b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36768/0 (socket says 172.21.15.44:36768) 2024-04-22T07:04:21.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f33bbfff640 1 -- 172.21.15.44:0/3676178664 learned_addr learned my addr 172.21.15.44:0/3676178664 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f33bb7fe640 1 --1- 172.21.15.44:0/3676178664 >> v1:172.21.15.70:6789/0 conn(0x7f33b404b090 0x7f33b40db830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:51708/0 (socket says 172.21.15.44:51708) 2024-04-22T07:04:21.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f3da3063640 1 --2- 172.21.15.44:0/2586965928 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d940dc390 0x7f3d94046230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.380+0000 7f3da3063640 1 --2- 172.21.15.44:0/2586965928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d940d5dc0 0x7f3d9404d7b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f3da3063640 1 -- 172.21.15.44:0/2586965928 >> 172.21.15.44:0/2586965928 conn(0x7f3d94051290 msgr2=0x7f3d940516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f3da3063640 1 -- 172.21.15.44:0/2586965928 shutdown_connections 2024-04-22T07:04:21.401 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f3da3063640 1 -- 172.21.15.44:0/2586965928 wait complete. 2024-04-22T07:04:21.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13cbfff640 1 --2- 172.21.15.44:0/255246690 >> v2:172.21.15.44:3300/0 conn(0x7f13c40508e0 0x7f13c40e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7fbb127fc640 1 -- 172.21.15.44:0/913267306 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08037070 con 0x7fbb14163160 2024-04-22T07:04:21.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7fbb127fc640 1 -- 172.21.15.44:0/913267306 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb08030710 con 0x7fbb14163160 2024-04-22T07:04:21.402 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7fbb127fc640 1 -- 172.21.15.44:0/913267306 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08030ae0 con 0x7fbb14163160 2024-04-22T07:04:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f33c0df9640 1 --2- 172.21.15.44:0/3676178664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33b400e2a0 0x7f33b4045330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f6639ffb640 1 -- 172.21.15.44:0/2318127264 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c035020 con 0x7f6634004a30 2024-04-22T07:04:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f6639ffb640 1 -- 172.21.15.44:0/2318127264 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f662c031830 con 0x7f6634004a30 2024-04-22T07:04:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f6639ffb640 1 -- 172.21.15.44:0/2318127264 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c031c00 con 0x7f6634004a30 2024-04-22T07:04:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f33baffd640 1 -- 172.21.15.44:0/3676178664 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192553583 0 0) 0x7f33b4045ed0 con 0x7f33b404b090 2024-04-22T07:04:21.403 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f33bbfff640 1 -- 172.21.15.44:0/3676178664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33b400e2a0 msgr2=0x7f33b4045330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f33bbfff640 1 --2- 172.21.15.44:0/3676178664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33b400e2a0 0x7f33b4045330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f33bbfff640 1 -- 172.21.15.44:0/3676178664 >> v1:172.21.15.70:6789/0 conn(0x7f33b404b090 legacy=0x7f33b40db830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f33bbfff640 1 -- 172.21.15.44:0/3676178664 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b4046450 con 0x7f33b40d5200 2024-04-22T07:04:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f53767fc640 1 --2- 172.21.15.44:0/926470264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53780c3480 0x7f537815fa20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f53780736f0 tx=0x7f536c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a3b82148902c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13d0e99640 1 -- 172.21.15.44:0/255246690 >> v2:172.21.15.44:3300/0 conn(0x7f13c40508e0 msgr2=0x7f13c40e2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13d0e99640 1 --2- 172.21.15.44:0/255246690 >> v2:172.21.15.44:3300/0 conn(0x7f13c40508e0 0x7f13c40e2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13d0e99640 1 -- 172.21.15.44:0/255246690 >> v1:172.21.15.44:6789/0 conn(0x7f13c40e2850 legacy=0x7f13c4049420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13d0e99640 1 -- 172.21.15.44:0/255246690 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c4007100 con 0x7f13c40d9040 2024-04-22T07:04:21.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13acc74640 1 -- 172.21.15.44:0/4180568215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13a004ba60 msgr2=0x7f13a004be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13acc74640 1 --2- 172.21.15.44:0/4180568215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13a004ba60 0x7f13a004be80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f13a802db50 tx=0x7f13a80a7a40 comp rx=0 tx=0).stop 2024-04-22T07:04:21.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f5375ffb640 1 -- 172.21.15.44:0/926470264 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c034070 con 0x7f53780c3480 2024-04-22T07:04:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f5375ffb640 1 -- 172.21.15.44:0/926470264 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536c030850 con 0x7f53780c3480 2024-04-22T07:04:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f5375ffb640 1 -- 172.21.15.44:0/926470264 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c034070 con 0x7f53780c3480 2024-04-22T07:04:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13acc74640 1 -- 172.21.15.44:0/4180568215 shutdown_connections 2024-04-22T07:04:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13acc74640 1 --2- 172.21.15.44:0/4180568215 >> v2:172.21.15.44:3300/0 conn(0x7f13a00e3090 0x7f13a00e34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13acc74640 1 --2- 172.21.15.44:0/4180568215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13a004ba60 0x7f13a004be80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.381+0000 7f13acc74640 1 -- 172.21.15.44:0/4180568215 >> 172.21.15.44:0/4180568215 conn(0x7f13a0050180 msgr2=0x7f13a0050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f13dccb3640 1 -- 172.21.15.44:0/147561741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d806c4f0 msgr2=0x7f13d815b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f13dccb3640 1 --2- 172.21.15.44:0/147561741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d806c4f0 0x7f13d815b4b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009dc0 tx=0x7f13d002bf10 comp rx=0 tx=0).stop 2024-04-22T07:04:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f13dccb3640 1 -- 172.21.15.44:0/147561741 shutdown_connections 2024-04-22T07:04:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f13dccb3640 1 --2- 172.21.15.44:0/147561741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d806c4f0 0x7f13d815b4b0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009dc0 tx=0x7f13d002bf10 comp rx=0 tx=0).stop 2024-04-22T07:04:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f13dccb3640 1 -- 172.21.15.44:0/147561741 >> 172.21.15.44:0/147561741 conn(0x7f13d8073470 msgr2=0x7f13d8073880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7fbb19e55640 1 -- 172.21.15.44:0/913267306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb14163160 msgr2=0x7fbb14072460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7fbb19e55640 1 --2- 172.21.15.44:0/913267306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb14163160 0x7fbb14072460 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbb14070c30 tx=0x7fbb0802b710 comp rx=0 tx=0).stop 2024-04-22T07:04:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f663bfff640 1 -- 172.21.15.44:0/2318127264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6634004a30 msgr2=0x7f663404bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f663bfff640 1 --2- 172.21.15.44:0/2318127264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6634004a30 0x7f663404bd80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f662c0097a0 tx=0x7f662c02b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f663bfff640 1 -- 172.21.15.44:0/2318127264 shutdown_connections 2024-04-22T07:04:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f663bfff640 1 --2- 172.21.15.44:0/2318127264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6634004a30 0x7f663404bd80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7fbb19e55640 1 -- 172.21.15.44:0/913267306 shutdown_connections 2024-04-22T07:04:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7fbb19e55640 1 --2- 172.21.15.44:0/913267306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb14163160 0x7fbb14072460 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7fbb19e55640 1 --2- 172.21.15.44:0/913267306 >> v2:172.21.15.44:3300/0 conn(0x7fbb141592d0 0x7fbb140716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7fbb19e55640 1 -- 172.21.15.44:0/913267306 >> 172.21.15.44:0/913267306 conn(0x7fbb1406b970 msgr2=0x7fbb1406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f663bfff640 1 --2- 172.21.15.44:0/2318127264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f66340d6f20 0x7f663404c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f663bfff640 1 -- 172.21.15.44:0/2318127264 >> 172.21.15.44:0/2318127264 conn(0x7f66340505d0 msgr2=0x7f66340509e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.409 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7fbb19e55640 1 -- 172.21.15.44:0/913267306 shutdown_connections 2024-04-22T07:04:21.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f13acc74640 1 -- 172.21.15.44:0/4180568215 shutdown_connections 2024-04-22T07:04:21.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f13acc74640 1 -- 172.21.15.44:0/4180568215 wait complete. 2024-04-22T07:04:21.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7fbb19e55640 1 -- 172.21.15.44:0/913267306 wait complete. 2024-04-22T07:04:21.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f663bfff640 1 -- 172.21.15.44:0/2318127264 shutdown_connections 2024-04-22T07:04:21.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f663bfff640 1 -- 172.21.15.44:0/2318127264 wait complete. 2024-04-22T07:04:21.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f33bbfff640 1 --2- 172.21.15.44:0/3676178664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b40d5200 0x7f33b404b620 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009870 tx=0x7f33b002ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df160689c1b9f736 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.382+0000 7f3da3063640 1 Processor -- start 2024-04-22T07:04:21.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f13dccb3640 1 -- 172.21.15.44:0/147561741 shutdown_connections 2024-04-22T07:04:21.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f3da3063640 1 -- start start 2024-04-22T07:04:21.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f13dccb3640 1 -- 172.21.15.44:0/147561741 wait complete. 2024-04-22T07:04:21.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f537d481640 1 -- 172.21.15.44:0/926470264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53780c3480 msgr2=0x7f537815fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.412 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f537d481640 1 --2- 172.21.15.44:0/926470264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53780c3480 0x7f537815fa20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f53780736f0 tx=0x7f536c02b990 comp rx=0 tx=0).stop 2024-04-22T07:04:21.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f13d0e99640 1 --2- 172.21.15.44:0/255246690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40d9040 0x7f13c40061f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13c4011d90 tx=0x7f13c002b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a091ee19172aad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f33baffd640 1 -- 172.21.15.44:0/3676178664 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b002f830 con 0x7f33b40d5200 2024-04-22T07:04:21.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f33baffd640 1 -- 172.21.15.44:0/3676178664 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b002f9d0 con 0x7f33b40d5200 2024-04-22T07:04:21.413 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f33baffd640 1 -- 172.21.15.44:0/3676178664 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b002fda0 con 0x7f33b40d5200 2024-04-22T07:04:21.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f13cb7fe640 1 -- 172.21.15.44:0/255246690 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0037070 con 0x7f13c40d9040 2024-04-22T07:04:21.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f13cb7fe640 1 -- 172.21.15.44:0/255246690 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c0030710 con 0x7f13c40d9040 2024-04-22T07:04:21.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f13cb7fe640 1 -- 172.21.15.44:0/255246690 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0030ae0 con 0x7f13c40d9040 2024-04-22T07:04:21.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f537d481640 1 -- 172.21.15.44:0/926470264 shutdown_connections 2024-04-22T07:04:21.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f537d481640 1 --2- 172.21.15.44:0/926470264 >> v2:172.21.15.70:3300/0 conn(0x7f53780c9c80 0x7f53780c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f537d481640 1 --2- 172.21.15.44:0/926470264 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53780c3480 0x7f537815fa20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f537d481640 1 --2- 172.21.15.44:0/926470264 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53781592c0 0x7f5378162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.383+0000 7f537d481640 1 -- 172.21.15.44:0/926470264 >> 172.21.15.44:0/926470264 conn(0x7f5378071300 msgr2=0x7f53780716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f33c15fa640 1 -- 172.21.15.44:0/3676178664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b40d5200 msgr2=0x7f33b404b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f33c15fa640 1 --2- 172.21.15.44:0/3676178664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b40d5200 0x7f33b404b620 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009870 tx=0x7f33b002ba10 comp rx=0 tx=0).stop 2024-04-22T07:04:21.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f33c15fa640 1 -- 172.21.15.44:0/3676178664 shutdown_connections 2024-04-22T07:04:21.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f663bfff640 1 Processor -- start 2024-04-22T07:04:21.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f13acc74640 1 Processor -- start 2024-04-22T07:04:21.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f33c15fa640 1 --2- 172.21.15.44:0/3676178664 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33b400e2a0 0x7f33b4045330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.417 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f663bfff640 1 -- start start 2024-04-22T07:04:21.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f13acc74640 1 -- start start 2024-04-22T07:04:21.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f33c15fa640 1 --2- 172.21.15.44:0/3676178664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b40d5200 0x7f33b404b620 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f537d481640 1 -- 172.21.15.44:0/926470264 shutdown_connections 2024-04-22T07:04:21.418 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f537d481640 1 -- 172.21.15.44:0/926470264 wait complete. 2024-04-22T07:04:21.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f33c15fa640 1 -- 172.21.15.44:0/3676178664 >> 172.21.15.44:0/3676178664 conn(0x7f33b4050180 msgr2=0x7f33b4050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7fbb19e55640 1 Processor -- start 2024-04-22T07:04:21.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f13d1e9b640 1 -- 172.21.15.44:0/255246690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40d9040 msgr2=0x7f13c40061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.419 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f13d1e9b640 1 --2- 172.21.15.44:0/255246690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40d9040 0x7f13c40061f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13c4011d90 tx=0x7f13c002b7c0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f3da3063640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d94047e60 0x7f3d940ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f3da3063640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d940d5dc0 0x7f3d9417c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f3da3063640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d940dc390 0x7f3d940df030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f3d9bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d940d5dc0 0x7f3d9417c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f3d9bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d940d5dc0 0x7f3d9417c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:58998/0 (socket says 172.21.15.44:58998) 2024-04-22T07:04:21.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f3da1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d940dc390 0x7f3d940df030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f3da1600640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d940dc390 0x7f3d940df030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37932/0 (socket says 172.21.15.44:37932) 2024-04-22T07:04:21.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3da0dff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d94047e60 0x7f3d940ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3da0dff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d94047e60 0x7f3d940ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36780/0 (socket says 172.21.15.44:36780) 2024-04-22T07:04:21.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.384+0000 7f13dccb3640 1 Processor -- start 2024-04-22T07:04:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f13dccb3640 1 -- start start 2024-04-22T07:04:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3d9bfff640 1 -- 172.21.15.44:0/525652193 learned_addr learned my addr 172.21.15.44:0/525652193 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3da3063640 1 -- 172.21.15.44:0/525652193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3d94047660 con 0x7f3d940dc390 2024-04-22T07:04:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3da3063640 1 -- 172.21.15.44:0/525652193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9404c520 con 0x7f3d94047e60 2024-04-22T07:04:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3da3063640 1 -- 172.21.15.44:0/525652193 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f3d94047940 con 0x7f3d940d5dc0 2024-04-22T07:04:21.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7fbb19e55640 1 -- start start 2024-04-22T07:04:21.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f33c15fa640 1 -- 172.21.15.44:0/3676178664 shutdown_connections 2024-04-22T07:04:21.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f13d1e9b640 1 -- 172.21.15.44:0/255246690 shutdown_connections 2024-04-22T07:04:21.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f13d1e9b640 1 --2- 172.21.15.44:0/255246690 >> v2:172.21.15.44:3300/0 conn(0x7f13c40508e0 0x7f13c40e2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f13d1e9b640 1 --2- 172.21.15.44:0/255246690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40d9040 0x7f13c40061f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f13d1e9b640 1 -- 172.21.15.44:0/255246690 >> 172.21.15.44:0/255246690 conn(0x7f13c4054580 msgr2=0x7f13c4054990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f13d1e9b640 1 -- 172.21.15.44:0/255246690 shutdown_connections 2024-04-22T07:04:21.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f13d1e9b640 1 -- 172.21.15.44:0/255246690 wait complete. 2024-04-22T07:04:21.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f33c15fa640 1 -- 172.21.15.44:0/3676178664 wait complete. 2024-04-22T07:04:21.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f663bfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6634004a30 0x7f6634174e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13acc74640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a004ba60 0x7f13a00dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13acc74640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13a00dd3c0 0x7f13a00e21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f663bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6634033910 0x7f66341685c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13acc74640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13a00e3090 0x7f13a0179570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13acc74640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0045980 con 0x7f13a004ba60 2024-04-22T07:04:21.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13acc74640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00456a0 con 0x7f13a00e3090 2024-04-22T07:04:21.425 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13acc74640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f13a0045c60 con 0x7f13a00dd3c0 2024-04-22T07:04:21.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13dccb3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d80d0730 0x7f13d80cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13dccb3640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d8154ba0 0x7f13d8072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13dccb3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13d80cd2e0 0x7f13d81f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13dccb3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13d80c6090 con 0x7f13d80d0730 2024-04-22T07:04:21.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13dccb3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f13d80c5db0 con 0x7f13d80cd2e0 2024-04-22T07:04:21.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13dccb3640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f13d80cae10 con 0x7f13d8154ba0 2024-04-22T07:04:21.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13a6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13a00e3090 0x7f13a0179570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13a6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13a00e3090 0x7f13a0179570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36786/0 (socket says 172.21.15.44:36786) 2024-04-22T07:04:21.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13a6d9d640 1 -- 172.21.15.44:0/3836638405 learned_addr learned my addr 172.21.15.44:0/3836638405 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13d6ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d8154ba0 0x7f13d8072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7fbb19e55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb140729a0 0x7fbb14201c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7fbb19e55640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb141592d0 0x7fbb141569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7fbb19e55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14163160 0x7fbb1415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7fbb19e55640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14072140 con 0x7fbb14163160 2024-04-22T07:04:21.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7fbb19e55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1415fbd0 con 0x7fbb140729a0 2024-04-22T07:04:21.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7fbb19e55640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbb14163580 con 0x7fbb141592d0 2024-04-22T07:04:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13a5d9b640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13a00dd3c0 0x7f13a00e21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13d80cd2e0 0x7f13d81f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13d7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13d80cd2e0 0x7f13d81f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36800/0 (socket says 172.21.15.44:36800) 2024-04-22T07:04:21.428 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13d7fff640 1 -- 172.21.15.44:0/2258115876 learned_addr learned my addr 172.21.15.44:0/2258115876 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14163160 0x7fbb1415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7fbb13fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14163160 0x7fbb1415a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37938/0 (socket says 172.21.15.44:37938) 2024-04-22T07:04:21.429 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7fbb13fff640 1 -- 172.21.15.44:0/2624855072 learned_addr learned my addr 172.21.15.44:0/2624855072 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f663bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66340d6f20 0x7f66340d8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f13d77fe640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d80d0730 0x7f13d80cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7fbb12ffd640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb141592d0 0x7fbb141569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f663bfff640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f66340e0ad0 con 0x7f6634033910 2024-04-22T07:04:21.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f663bfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f66340e0950 con 0x7f66340d6f20 2024-04-22T07:04:21.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66340d6f20 0x7f66340d8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f663b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66340d6f20 0x7f66340d8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36812/0 (socket says 172.21.15.44:36812) 2024-04-22T07:04:21.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f663b7fe640 1 -- 172.21.15.44:0/4111515487 learned_addr learned my addr 172.21.15.44:0/4111515487 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f663affd640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6634004a30 0x7f6634174e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.431 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6634168150 con 0x7f6634004a30 2024-04-22T07:04:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f13d6ffd640 1 -- 172.21.15.44:0/2258115876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13d80cd2e0 msgr2=0x7f13d81f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f13d6ffd640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13d80cd2e0 0x7f13d81f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f33c15fa640 1 Processor -- start 2024-04-22T07:04:21.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f13d6ffd640 1 -- 172.21.15.44:0/2258115876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d80d0730 msgr2=0x7f13d80cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f13d6ffd640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13d80d0730 0x7f13d80cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f13d1e9b640 1 Processor -- start 2024-04-22T07:04:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f13d6ffd640 1 -- 172.21.15.44:0/2258115876 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d80cc710 con 0x7f13d8154ba0 2024-04-22T07:04:21.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f13d1e9b640 1 -- start start 2024-04-22T07:04:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f13d6ffd640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d8154ba0 0x7f13d8072570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f13d80d13f0 tx=0x7f13d0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f537d481640 1 Processor -- start 2024-04-22T07:04:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f537d481640 1 -- start start 2024-04-22T07:04:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f537d481640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53780c3480 0x7f5378157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f537d481640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53780c9c80 0x7f53781545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f537d481640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53781592c0 0x7f5378154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.434 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f5376ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53780c3480 0x7f5378157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f5376ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53780c3480 0x7f5378157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:59034/0 (socket says 172.21.15.44:59034) 2024-04-22T07:04:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f53767fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53780c9c80 0x7f53781545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f53767fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53780c9c80 0x7f53781545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36814/0 (socket says 172.21.15.44:36814) 2024-04-22T07:04:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f5376ffd640 1 -- 172.21.15.44:0/3075125521 learned_addr learned my addr 172.21.15.44:0/3075125521 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f537d481640 1 -- 172.21.15.44:0/3075125521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f53781631f0 con 0x7f53781592c0 2024-04-22T07:04:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f537d481640 1 -- 172.21.15.44:0/3075125521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5378162f10 con 0x7f53780c9c80 2024-04-22T07:04:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f537d481640 1 -- 172.21.15.44:0/3075125521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f537815f220 con 0x7f53780c3480 2024-04-22T07:04:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3da1600640 1 -- 172.21.15.44:0/525652193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d940d5dc0 msgr2=0x7f3d9417c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3da1600640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3d940d5dc0 0x7f3d9417c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.435 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3da1600640 1 -- 172.21.15.44:0/525652193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d94047e60 msgr2=0x7f3d940ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3da1600640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d94047e60 0x7f3d940ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.385+0000 7f3da1600640 1 -- 172.21.15.44:0/525652193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9404fbf0 con 0x7f3d940dc390 2024-04-22T07:04:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f3da1600640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d940dc390 0x7f3d940df030 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000e7f0 tx=0x7f3d9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f3da0dff640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d94047e60 0x7f3d940ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f3d997fa640 1 -- 172.21.15.44:0/525652193 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90017070 con 0x7f3d940dc390 2024-04-22T07:04:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f3d997fa640 1 -- 172.21.15.44:0/525652193 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9000c730 con 0x7f3d940dc390 2024-04-22T07:04:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.386+0000 7f3d997fa640 1 -- 172.21.15.44:0/525652193 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000cb60 con 0x7f3d940dc390 2024-04-22T07:04:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f3da3063640 1 -- 172.21.15.44:0/525652193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d94051050 con 0x7f3d940dc390 2024-04-22T07:04:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.387+0000 7f3da3063640 1 -- 172.21.15.44:0/525652193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9417d1d0 con 0x7f3d940dc390 2024-04-22T07:04:21.436 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7fbb137fe640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb140729a0 0x7fbb14201c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f663a7fc640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6634033910 0x7f66341685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f663b7fe640 1 -- 172.21.15.44:0/4111515487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6634004a30 msgr2=0x7f6634174e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f663b7fe640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6634004a30 0x7f6634174e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f663b7fe640 1 -- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6634033910 msgr2=0x7f66341685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f663b7fe640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6634033910 0x7f66341685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f663b7fe640 1 -- 172.21.15.44:0/4111515487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66340d7b20 con 0x7f66340d6f20 2024-04-22T07:04:21.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13a6d9d640 1 -- 172.21.15.44:0/3836638405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13a00dd3c0 msgr2=0x7f13a00e21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13a6d9d640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13a00dd3c0 0x7f13a00e21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13a6d9d640 1 -- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a004ba60 msgr2=0x7f13a00dbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13a6d9d640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13a004ba60 0x7f13a00dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13a6d9d640 1 -- 172.21.15.44:0/3836638405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a00da8c0 con 0x7f13a00e3090 2024-04-22T07:04:21.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f663a7fc640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6634033910 0x7f66341685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:21.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f663b7fe640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66340d6f20 0x7f66340d8e80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f663c068990 tx=0x7f663c0695e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.438 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f53777fe640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53781592c0 0x7f5378154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13bffff640 1 -- 172.21.15.44:0/2258115876 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0044030 con 0x7f13d8154ba0 2024-04-22T07:04:21.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13bffff640 1 -- 172.21.15.44:0/2258115876 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d0003990 con 0x7f13d8154ba0 2024-04-22T07:04:21.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13bffff640 1 -- 172.21.15.44:0/2258115876 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0039070 con 0x7f13d8154ba0 2024-04-22T07:04:21.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13dccb3640 1 -- 172.21.15.44:0/2258115876 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d80cd840 con 0x7f13d8154ba0 2024-04-22T07:04:21.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13dccb3640 1 -- 172.21.15.44:0/2258115876 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d81f4dd0 con 0x7f13d8154ba0 2024-04-22T07:04:21.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f66237fe640 1 -- 172.21.15.44:0/4111515487 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c06bd80 con 0x7f66340d6f20 2024-04-22T07:04:21.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f13a6d9d640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13a00e3090 0x7f13a0179570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f139800e870 tx=0x7f139800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.388+0000 7f33c15fa640 1 -- start start 2024-04-22T07:04:21.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33c15fa640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33b400e2a0 0x7f33b4179580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66340d7e00 con 0x7f66340d6f20 2024-04-22T07:04:21.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66340d83c0 con 0x7f66340d6f20 2024-04-22T07:04:21.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33c15fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b404b090 0x7f33b40cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13d1e9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40508e0 0x7f13c4052930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13d1e9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40d9040 0x7f13c40dace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13d1e9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40e2850 0x7f13c40db220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13d1e9b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f13c4050d30 con 0x7f13c40d9040 2024-04-22T07:04:21.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13d1e9b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f13c40076c0 con 0x7f13c40e2850 2024-04-22T07:04:21.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13d1e9b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f13c40d9420 con 0x7f13c40508e0 2024-04-22T07:04:21.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f66237fe640 1 -- 172.21.15.44:0/4111515487 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663c0922b0 con 0x7f66340d6f20 2024-04-22T07:04:21.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33c15fa640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40d5200 0x7f33b40491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7fbb13fff640 1 -- 172.21.15.44:0/2624855072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb141592d0 msgr2=0x7fbb141569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7fbb13fff640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbb141592d0 0x7fbb141569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7fbb13fff640 1 -- 172.21.15.44:0/2624855072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb140729a0 msgr2=0x7fbb14201c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7fbb13fff640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb140729a0 0x7fbb14201c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7fbb13fff640 1 -- 172.21.15.44:0/2624855072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb141626e0 con 0x7fbb14163160 2024-04-22T07:04:21.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f66237fe640 1 -- 172.21.15.44:0/4111515487 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c092660 con 0x7f66340d6f20 2024-04-22T07:04:21.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f1396ffd640 1 -- 172.21.15.44:0/3836638405 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398017070 con 0x7f13a00e3090 2024-04-22T07:04:21.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f1396ffd640 1 -- 172.21.15.44:0/3836638405 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139800c940 con 0x7f13a00e3090 2024-04-22T07:04:21.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f1396ffd640 1 -- 172.21.15.44:0/3836638405 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139800cd30 con 0x7f13a00e3090 2024-04-22T07:04:21.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a00e0340 con 0x7f13a00e3090 2024-04-22T07:04:21.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a00e2710 con 0x7f13a00e3090 2024-04-22T07:04:21.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f13bdffb640 1 -- 172.21.15.44:0/2258115876 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13d80c7780 con 0x7f13d8154ba0 2024-04-22T07:04:21.444 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b404b090 0x7f33b40cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b404b090 0x7f33b40cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36834/0 (socket says 172.21.15.44:36834) 2024-04-22T07:04:21.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33b400e2a0 0x7f33b4179580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33b400e2a0 0x7f33b4179580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:59040/0 (socket says 172.21.15.44:59040) 2024-04-22T07:04:21.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33bb7fe640 1 -- 172.21.15.44:0/3846063616 learned_addr learned my addr 172.21.15.44:0/3846063616 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33c15fa640 1 -- 172.21.15.44:0/3846063616 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33b4045870 con 0x7f33b40d5200 2024-04-22T07:04:21.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33c15fa640 1 -- 172.21.15.44:0/3846063616 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f33b4046a10 con 0x7f33b404b090 2024-04-22T07:04:21.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f33c15fa640 1 -- 172.21.15.44:0/3846063616 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f33b40459f0 con 0x7f33b400e2a0 2024-04-22T07:04:21.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13d169a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40e2850 0x7f13c40db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13d169a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40e2850 0x7f13c40db220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36818/0 (socket says 172.21.15.44:36818) 2024-04-22T07:04:21.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13d169a640 1 -- 172.21.15.44:0/4134883390 learned_addr learned my addr 172.21.15.44:0/4134883390 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f13d0e99640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40508e0 0x7f13c4052930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f3d86ffd640 1 -- 172.21.15.44:0/525652193 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d940cfe40 con 0x7f3d940dc390 2024-04-22T07:04:21.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f5376ffd640 1 -- 172.21.15.44:0/3075125521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53780c9c80 msgr2=0x7f53781545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f5376ffd640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53780c9c80 0x7f53781545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f5376ffd640 1 -- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53781592c0 msgr2=0x7f5378154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.389+0000 7f5376ffd640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f53781592c0 0x7f5378154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f5376ffd640 1 -- 172.21.15.44:0/3075125521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53780cda00 con 0x7f53780c3480 2024-04-22T07:04:21.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f13d169a640 1 -- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40508e0 msgr2=0x7f13c4052930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f13d169a640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13c40508e0 0x7f13c4052930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f13d169a640 1 -- 172.21.15.44:0/4134883390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40d9040 msgr2=0x7f13c40dace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f13d169a640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f13c40d9040 0x7f13c40dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f13d169a640 1 -- 172.21.15.44:0/4134883390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c40d9f90 con 0x7f13c40e2850 2024-04-22T07:04:21.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f3d997fa640 1 -- 172.21.15.44:0/525652193 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d9001a070 con 0x7f3d940dc390 2024-04-22T07:04:21.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66340d8600 con 0x7f66340d6f20 2024-04-22T07:04:21.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7fbb13fff640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14163160 0x7fbb1415a110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00caf0 tx=0x7fbb0c0020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f13bffff640 1 -- 172.21.15.44:0/2258115876 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13d0040660 con 0x7f13d8154ba0 2024-04-22T07:04:21.449 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7f13d169a640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40e2850 0x7f13c40db220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13cc06a210 tx=0x7f13cc069f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.390+0000 7fbafbfff640 1 -- 172.21.15.44:0/2624855072 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c018070 con 0x7fbb14163160 2024-04-22T07:04:21.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7fbafbfff640 1 -- 172.21.15.44:0/2624855072 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0c00a700 con 0x7fbb14163160 2024-04-22T07:04:21.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7fbafbfff640 1 -- 172.21.15.44:0/2624855072 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c018070 con 0x7fbb14163160 2024-04-22T07:04:21.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f138c001030 con 0x7f13a00e3090 2024-04-22T07:04:21.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7fbb19e55640 1 -- 172.21.15.44:0/2624855072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1415c500 con 0x7fbb14163160 2024-04-22T07:04:21.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7fbb19e55640 1 -- 172.21.15.44:0/2624855072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb14159bf0 con 0x7fbb14163160 2024-04-22T07:04:21.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f5376ffd640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53780c3480 0x7f5378157f80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f536800d620 tx=0x7f536800dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f66237fe640 1 -- 172.21.15.44:0/4111515487 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f663c092800 con 0x7f66340d6f20 2024-04-22T07:04:21.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f33c0df9640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40d5200 0x7f33b40491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f33bb7fe640 1 -- 172.21.15.44:0/3846063616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33b400e2a0 msgr2=0x7f33b4179580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f33bb7fe640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33b400e2a0 0x7f33b4179580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f33bb7fe640 1 -- 172.21.15.44:0/3846063616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40d5200 msgr2=0x7f33b40491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f33bb7fe640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40d5200 0x7f33b40491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f33bb7fe640 1 -- 172.21.15.44:0/3846063616 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b40cc690 con 0x7f33b404b090 2024-04-22T07:04:21.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f33c0df9640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33b40d5200 0x7f33b40491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:21.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f13c97fa640 1 -- 172.21.15.44:0/4134883390 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc090070 con 0x7f13c40e2850 2024-04-22T07:04:21.452 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f13c97fa640 1 -- 172.21.15.44:0/4134883390 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc06ce50 con 0x7f13c40e2850 2024-04-22T07:04:21.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c4053280 con 0x7f13c40e2850 2024-04-22T07:04:21.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f66237fe640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f662806c8f0 0x7f662806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f13bffff640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13c406c970 0x7f13c406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f13bffff640 1 -- 172.21.15.44:0/2258115876 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f13d00ae180 con 0x7f13d8154ba0 2024-04-22T07:04:21.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f13d77fe640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13c406c970 0x7f13c406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f13bffff640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.70:6816/1770474213,v1:172.21.15.70:6818/1770474213] conn(0x7f13c4074bf0 0x7f13c4077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f13bffff640 1 -- 172.21.15.44:0/2258115876 --> [v2:172.21.15.70:6816/1770474213,v1:172.21.15.70:6818/1770474213] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13d002ee50 con 0x7f13c4074bf0 2024-04-22T07:04:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f13bffff640 1 -- 172.21.15.44:0/2258115876 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f13d00b5050 con 0x7f13d8154ba0 2024-04-22T07:04:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f13d7fff640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.70:6816/1770474213,v1:172.21.15.70:6818/1770474213] conn(0x7f13c4074bf0 0x7f13c4077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f13d77fe640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13c406c970 0x7f13c406ee30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005880 tx=0x7f13c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f13c97fa640 1 -- 172.21.15.44:0/4134883390 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0625a0 con 0x7f13c40e2850 2024-04-22T07:04:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c40da200 con 0x7f13c40e2850 2024-04-22T07:04:21.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f66237fe640 1 -- 172.21.15.44:0/4111515487 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f663c107430 con 0x7f66340d6f20 2024-04-22T07:04:21.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f66237fe640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:6800/3148242873,v1:172.21.15.44:6801/3148242873] conn(0x7f6628074b10 0x7f6628076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7fbaf9ffb640 1 -- 172.21.15.44:0/2624855072 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb140ccf50 con 0x7fbb14163160 2024-04-22T07:04:21.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f535f7fe640 1 -- 172.21.15.44:0/3075125521 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536800b620 con 0x7f53780c3480 2024-04-22T07:04:21.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f535f7fe640 1 -- 172.21.15.44:0/3075125521 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5368019070 con 0x7f53780c3480 2024-04-22T07:04:21.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f535f7fe640 1 -- 172.21.15.44:0/3075125521 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536800ba00 con 0x7f53780c3480 2024-04-22T07:04:21.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f33bb7fe640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b404b090 0x7f33b40cf160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f33a800c4d0 tx=0x7f33a800c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f3d997fa640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3d8006c9f0 0x7f3d8006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f3da0dff640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3d8006c9f0 0x7f3d8006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.391+0000 7f3d997fa640 1 -- 172.21.15.44:0/525652193 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3d9008b8a0 con 0x7f3d940dc390 2024-04-22T07:04:21.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f3d997fa640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.70:6817/1512457027,v1:172.21.15.70:6819/1512457027] conn(0x7f3d80074c10 0x7f3d80077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f3d997fa640 1 -- 172.21.15.44:0/525652193 --> [v2:172.21.15.70:6817/1512457027,v1:172.21.15.70:6819/1512457027] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d800776e0 con 0x7f3d80074c10 2024-04-22T07:04:21.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f3d997fa640 1 -- 172.21.15.44:0/525652193 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3d900902a0 con 0x7f3d940dc390 2024-04-22T07:04:21.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f3d9bfff640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.70:6817/1512457027,v1:172.21.15.70:6819/1512457027] conn(0x7f3d80074c10 0x7f3d80077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f537d481640 1 -- 172.21.15.44:0/3075125521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537815ce50 con 0x7f53780c3480 2024-04-22T07:04:21.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f3da0dff640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3d8006c9f0 0x7f3d8006eeb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400ef50 tx=0x7f3d9c06e030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f33b8ff9640 1 -- 172.21.15.44:0/3846063616 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8018070 con 0x7f33b404b090 2024-04-22T07:04:21.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f33b8ff9640 1 -- 172.21.15.44:0/3846063616 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a80074e0 con 0x7f33b404b090 2024-04-22T07:04:21.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13c4006450 con 0x7f13c40e2850 2024-04-22T07:04:21.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f33b8ff9640 1 -- 172.21.15.44:0/3846063616 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8004a20 con 0x7f33b404b090 2024-04-22T07:04:21.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f13d7fff640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.70:6816/1770474213,v1:172.21.15.70:6818/1770474213] conn(0x7f13c4074bf0 0x7f13c4077010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f537d481640 1 -- 172.21.15.44:0/3075125521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5378161d90 con 0x7f53780c3480 2024-04-22T07:04:21.460 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f1396ffd640 1 -- 172.21.15.44:0/3836638405 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f139801a070 con 0x7f13a00e3090 2024-04-22T07:04:21.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.392+0000 7f66237fe640 1 -- 172.21.15.44:0/4111515487 --> [v2:172.21.15.44:6800/3148242873,v1:172.21.15.44:6801/3148242873] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66280775e0 con 0x7f6628074b10 2024-04-22T07:04:21.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f33c15fa640 1 -- 172.21.15.44:0/3846063616 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b40cc3b0 con 0x7f33b404b090 2024-04-22T07:04:21.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f66237fe640 1 -- 172.21.15.44:0/4111515487 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f663c10e050 con 0x7f66340d6f20 2024-04-22T07:04:21.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f33c15fa640 1 -- 172.21.15.44:0/3846063616 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b404a480 con 0x7f33b404b090 2024-04-22T07:04:21.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f663a7fc640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:6800/3148242873,v1:172.21.15.44:6801/3148242873] conn(0x7f6628074b10 0x7f6628076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f3d9bfff640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.70:6817/1512457027,v1:172.21.15.70:6819/1512457027] conn(0x7f3d80074c10 0x7f3d80077030 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.394+0000 7f13bffff640 1 -- 172.21.15.44:0/2258115876 <== osd.4 v2:172.21.15.70:6816/1770474213 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13d002ee50 con 0x7f13c4074bf0 2024-04-22T07:04:21.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.394+0000 7f13c97fa640 1 -- 172.21.15.44:0/4134883390 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13cc093070 con 0x7f13c40e2850 2024-04-22T07:04:21.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.394+0000 7f1396ffd640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f138806c940 0x7f138806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.394+0000 7f1396ffd640 1 -- 172.21.15.44:0/3836638405 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f139808c250 con 0x7f13a00e3090 2024-04-22T07:04:21.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.393+0000 7f663affd640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f662806c8f0 0x7f662806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.394+0000 7f663a7fc640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:6800/3148242873,v1:172.21.15.44:6801/3148242873] conn(0x7f6628074b10 0x7f6628076f30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.394+0000 7f1396ffd640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:6816/2173398368,v1:172.21.15.44:6817/2173398368] conn(0x7f1388074bc0 0x7f1388076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.394+0000 7f1396ffd640 1 -- 172.21.15.44:0/3836638405 --> [v2:172.21.15.44:6816/2173398368,v1:172.21.15.44:6817/2173398368] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1388077690 con 0x7f1388074bc0 2024-04-22T07:04:21.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.394+0000 7f1396ffd640 1 -- 172.21.15.44:0/3836638405 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f139808c730 con 0x7f13a00e3090 2024-04-22T07:04:21.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.394+0000 7f13a659c640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f138806c940 0x7f138806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f3d997fa640 1 -- 172.21.15.44:0/525652193 <== osd.5 v2:172.21.15.70:6817/1512457027 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d800776e0 con 0x7f3d80074c10 2024-04-22T07:04:21.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f33a67fc640 1 -- 172.21.15.44:0/3846063616 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33b40052c0 con 0x7f33b404b090 2024-04-22T07:04:21.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f13a5d9b640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:6816/2173398368,v1:172.21.15.44:6817/2173398368] conn(0x7f1388074bc0 0x7f1388076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f537d481640 1 -- 172.21.15.44:0/3075125521 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5378161ba0 con 0x7f53780c3480 2024-04-22T07:04:21.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f13a659c640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f138806c940 0x7f138806ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f139c000f60 tx=0x7f139c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7fbafbfff640 1 -- 172.21.15.44:0/2624855072 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb0c00a8b0 con 0x7fbb14163160 2024-04-22T07:04:21.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f13c97fa640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c9c0 0x7f13b006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f13a5d9b640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:6816/2173398368,v1:172.21.15.44:6817/2173398368] conn(0x7f1388074bc0 0x7f1388076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f13c97fa640 1 -- 172.21.15.44:0/4134883390 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f13cc107710 con 0x7f13c40e2850 2024-04-22T07:04:21.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f13c97fa640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:6808/3781953159,v1:172.21.15.70:6809/3781953159] conn(0x7f13b0074c40 0x7f13b0077060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f13c97fa640 1 -- 172.21.15.44:0/4134883390 --> [v2:172.21.15.70:6808/3781953159,v1:172.21.15.70:6809/3781953159] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13b0077710 con 0x7f13b0074c40 2024-04-22T07:04:21.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7f13c97fa640 1 -- 172.21.15.44:0/4134883390 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f13cc10e050 con 0x7f13c40e2850 2024-04-22T07:04:21.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7fbafbfff640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbb0006caf0 0x7fbb0006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.395+0000 7fbafbfff640 1 -- 172.21.15.44:0/2624855072 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbb0c08af70 con 0x7fbb14163160 2024-04-22T07:04:21.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.396+0000 7fbafbfff640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:6824/1806128924,v1:172.21.15.44:6825/1806128924] conn(0x7fbb00074cf0 0x7fbb00077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.396+0000 7fbafbfff640 1 -- 172.21.15.44:0/2624855072 --> [v2:172.21.15.44:6824/1806128924,v1:172.21.15.44:6825/1806128924] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb000777c0 con 0x7fbb00074cf0 2024-04-22T07:04:21.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.396+0000 7fbafbfff640 1 -- 172.21.15.44:0/2624855072 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c054ed0 con 0x7fbb14163160 2024-04-22T07:04:21.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.394+0000 7f663affd640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f662806c8f0 0x7f662806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6630006fd0 tx=0x7f66300090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.396+0000 7f66237fe640 1 -- 172.21.15.44:0/4111515487 <== osd.2 v2:172.21.15.44:6800/3148242873 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f66280775e0 con 0x7f6628074b10 2024-04-22T07:04:21.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.396+0000 7fbb12ffd640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:6824/1806128924,v1:172.21.15.44:6825/1806128924] conn(0x7fbb00074cf0 0x7fbb00077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.396+0000 7fbb137fe640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbb0006caf0 0x7fbb0006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.396+0000 7f13d0e99640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c9c0 0x7f13b006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.396+0000 7f1396ffd640 1 -- 172.21.15.44:0/3836638405 <== osd.3 v2:172.21.15.44:6816/2173398368 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1388077690 con 0x7f1388074bc0 2024-04-22T07:04:21.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.396+0000 7fbb12ffd640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:6824/1806128924,v1:172.21.15.44:6825/1806128924] conn(0x7fbb00074cf0 0x7fbb00077110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.397+0000 7f13d0e99640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c9c0 0x7f13b006ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f13c0034000 tx=0x7f13c0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.397+0000 7f535f7fe640 1 -- 172.21.15.44:0/3075125521 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f536800bba0 con 0x7f53780c3480 2024-04-22T07:04:21.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.397+0000 7f13cbfff640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:6808/3781953159,v1:172.21.15.70:6809/3781953159] conn(0x7f13b0074c40 0x7f13b0077060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.397+0000 7fbb137fe640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbb0006caf0 0x7fbb0006efb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb04006fd0 tx=0x7fbb04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.397+0000 7f535f7fe640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f536406c9f0 0x7f536406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7f535f7fe640 1 -- 172.21.15.44:0/3075125521 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f536808bda0 con 0x7f53780c3480 2024-04-22T07:04:21.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7fbafbfff640 1 -- 172.21.15.44:0/2624855072 <== osd.0 v2:172.21.15.44:6824/1806128924 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb000777c0 con 0x7fbb00074cf0 2024-04-22T07:04:21.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7f535f7fe640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:6808/2837839457,v1:172.21.15.44:6809/2837839457] conn(0x7f5364074c70 0x7f5364077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7f535f7fe640 1 -- 172.21.15.44:0/3075125521 --> [v2:172.21.15.44:6808/2837839457,v1:172.21.15.44:6809/2837839457] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5368001d30 con 0x7f5364074c70 2024-04-22T07:04:21.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7f535f7fe640 1 -- 172.21.15.44:0/3075125521 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f536808e400 con 0x7f53780c3480 2024-04-22T07:04:21.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7f53777fe640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:6808/2837839457,v1:172.21.15.44:6809/2837839457] conn(0x7f5364074c70 0x7f5364077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.397+0000 7f33b8ff9640 1 -- 172.21.15.44:0/3846063616 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33a801b070 con 0x7f33b404b090 2024-04-22T07:04:21.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.397+0000 7f33b8ff9640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c9c0 0x7f33a006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7f33bbfff640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c9c0 0x7f33a006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7f33b8ff9640 1 -- 172.21.15.44:0/3846063616 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f33a808afa0 con 0x7f33b404b090 2024-04-22T07:04:21.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7f33b8ff9640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.70:6800/921795404,v1:172.21.15.70:6801/921795404] conn(0x7f33a0074be0 0x7f33a0077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7f33b8ff9640 1 -- 172.21.15.44:0/3846063616 --> [v2:172.21.15.70:6800/921795404,v1:172.21.15.70:6801/921795404] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f33a801ce50 con 0x7f33a0074be0 2024-04-22T07:04:21.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.398+0000 7f33b8ff9640 1 -- 172.21.15.44:0/3846063616 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f33a808d600 con 0x7f33b404b090 2024-04-22T07:04:21.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.399+0000 7f13cbfff640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:6808/3781953159,v1:172.21.15.70:6809/3781953159] conn(0x7f13b0074c40 0x7f13b0077060 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.399+0000 7f53777fe640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:6808/2837839457,v1:172.21.15.44:6809/2837839457] conn(0x7f5364074c70 0x7f5364077090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.399+0000 7f33c0df9640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.70:6800/921795404,v1:172.21.15.70:6801/921795404] conn(0x7f33a0074be0 0x7f33a0077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.399+0000 7f53767fc640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f536406c9f0 0x7f536406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.399+0000 7f33bbfff640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c9c0 0x7f33a006ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33b0036000 tx=0x7f33b0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.399+0000 7f53767fc640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f536406c9f0 0x7f536406eeb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f536c036000 tx=0x7f536c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.399+0000 7f13c97fa640 1 -- 172.21.15.44:0/4134883390 <== osd.7 v2:172.21.15.70:6808/3781953159 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13b0077710 con 0x7f13b0074c40 2024-04-22T07:04:21.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.399+0000 7f33c0df9640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.70:6800/921795404,v1:172.21.15.70:6801/921795404] conn(0x7f33a0074be0 0x7f33a0077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.400+0000 7f535f7fe640 1 -- 172.21.15.44:0/3075125521 <== osd.1 v2:172.21.15.44:6808/2837839457 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5368001d30 con 0x7f5364074c70 2024-04-22T07:04:21.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.400+0000 7f33b8ff9640 1 -- 172.21.15.44:0/3846063616 <== osd.6 v2:172.21.15.70:6800/921795404 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f33a801ce50 con 0x7f33a0074be0 2024-04-22T07:04:21.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.424+0000 7fbaf9ffb640 1 -- 172.21.15.44:0/2624855072 --> [v2:172.21.15.44:6824/1806128924,v1:172.21.15.44:6825/1806128924] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb14072780 con 0x7fbb00074cf0 2024-04-22T07:04:21.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.425+0000 7fbafbfff640 1 -- 172.21.15.44:0/2624855072 <== osd.0 v2:172.21.15.44:6824/1806128924 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbb14072780 con 0x7fbb00074cf0 2024-04-22T07:04:21.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.425+0000 7fbb19e55640 1 -- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:6824/1806128924,v1:172.21.15.44:6825/1806128924] conn(0x7fbb00074cf0 msgr2=0x7fbb00077110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.425+0000 7fbb19e55640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:6824/1806128924,v1:172.21.15.44:6825/1806128924] conn(0x7fbb00074cf0 0x7fbb00077110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.426+0000 7fbb19e55640 1 -- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbb0006caf0 msgr2=0x7fbb0006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.426+0000 7fbb19e55640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbb0006caf0 0x7fbb0006efb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb04006fd0 tx=0x7fbb04008040 comp rx=0 tx=0).stop 2024-04-22T07:04:21.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.426+0000 7fbb19e55640 1 -- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14163160 msgr2=0x7fbb1415a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.426+0000 7fbb19e55640 1 --2- 172.21.15.44:0/2624855072 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbb14163160 0x7fbb1415a110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00caf0 tx=0x7fbb0c0020a0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.426+0000 7fbb13fff640 1 -- 172.21.15.44:0/2624855072 reap_dead start 2024-04-22T07:04:21.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.426+0000 7fbb19e55640 1 -- 172.21.15.44:0/2624855072 shutdown_connections 2024-04-22T07:04:21.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.426+0000 7fbb19e55640 1 -- 172.21.15.44:0/2624855072 >> 172.21.15.44:0/2624855072 conn(0x7fbb1406b970 msgr2=0x7fbb1415d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.426+0000 7fbb19e55640 1 -- 172.21.15.44:0/2624855072 shutdown_connections 2024-04-22T07:04:21.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.426+0000 7fbb19e55640 1 -- 172.21.15.44:0/2624855072 wait complete. 2024-04-22T07:04:21.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.429+0000 7f13bdffb640 1 -- 172.21.15.44:0/2258115876 --> [v2:172.21.15.70:6816/1770474213,v1:172.21.15.70:6818/1770474213] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13d8068b00 con 0x7f13c4074bf0 2024-04-22T07:04:21.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.429+0000 7f3d86ffd640 1 -- 172.21.15.44:0/525652193 --> [v2:172.21.15.70:6817/1512457027,v1:172.21.15.70:6819/1512457027] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d940cb120 con 0x7f3d80074c10 2024-04-22T07:04:21.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.430+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 --> [v2:172.21.15.70:6808/3781953159,v1:172.21.15.70:6809/3781953159] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13c4005710 con 0x7f13b0074c40 2024-04-22T07:04:21.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.430+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 --> [v2:172.21.15.44:6800/3148242873,v1:172.21.15.44:6801/3148242873] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6634005710 con 0x7f6628074b10 2024-04-22T07:04:21.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.430+0000 7f13bffff640 1 -- 172.21.15.44:0/2258115876 <== osd.4 v2:172.21.15.70:6816/1770474213 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f13d8068b00 con 0x7f13c4074bf0 2024-04-22T07:04:21.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.430+0000 7f13c97fa640 1 -- 172.21.15.44:0/4134883390 <== osd.7 v2:172.21.15.70:6808/3781953159 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f13c4005710 con 0x7f13b0074c40 2024-04-22T07:04:21.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.430+0000 7f66237fe640 1 -- 172.21.15.44:0/4111515487 <== osd.2 v2:172.21.15.44:6800/3148242873 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6634005710 con 0x7f6628074b10 2024-04-22T07:04:21.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f3d997fa640 1 -- 172.21.15.44:0/525652193 <== osd.5 v2:172.21.15.70:6817/1512457027 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3d940cb120 con 0x7f3d80074c10 2024-04-22T07:04:21.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:6808/3781953159,v1:172.21.15.70:6809/3781953159] conn(0x7f13b0074c40 msgr2=0x7f13b0077060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d1e9b640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:6808/3781953159,v1:172.21.15.70:6809/3781953159] conn(0x7f13b0074c40 0x7f13b0077060 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:6800/3148242873,v1:172.21.15.44:6801/3148242873] conn(0x7f6628074b10 msgr2=0x7f6628076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f663bfff640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:6800/3148242873,v1:172.21.15.44:6801/3148242873] conn(0x7f6628074b10 0x7f6628076f30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.481 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c9c0 msgr2=0x7f13b006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d1e9b640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13b006c9c0 0x7f13b006ee80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f13c0034000 tx=0x7f13c0008030 comp rx=0 tx=0).stop 2024-04-22T07:04:21.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40e2850 msgr2=0x7f13c40db220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d1e9b640 1 --2- 172.21.15.44:0/4134883390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c40e2850 0x7f13c40db220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13cc06a210 tx=0x7f13cc069f20 comp rx=0 tx=0).stop 2024-04-22T07:04:21.482 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13dccb3640 1 -- 172.21.15.44:0/2258115876 >> [v2:172.21.15.70:6816/1770474213,v1:172.21.15.70:6818/1770474213] conn(0x7f13c4074bf0 msgr2=0x7f13c4077010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d169a640 1 -- 172.21.15.44:0/4134883390 reap_dead start 2024-04-22T07:04:21.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 shutdown_connections 2024-04-22T07:04:21.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 >> 172.21.15.44:0/4134883390 conn(0x7f13c4054580 msgr2=0x7f13c404f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13dccb3640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.70:6816/1770474213,v1:172.21.15.70:6818/1770474213] conn(0x7f13c4074bf0 0x7f13c4077010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 shutdown_connections 2024-04-22T07:04:21.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f13d1e9b640 1 -- 172.21.15.44:0/4134883390 wait complete. 2024-04-22T07:04:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f662806c8f0 msgr2=0x7f662806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f663bfff640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f662806c8f0 0x7f662806edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6630006fd0 tx=0x7f66300090e0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66340d6f20 msgr2=0x7f66340d8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f663bfff640 1 --2- 172.21.15.44:0/4111515487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66340d6f20 0x7f66340d8e80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f663c068990 tx=0x7f663c0695e0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f3d86ffd640 1 -- 172.21.15.44:0/525652193 >> [v2:172.21.15.70:6817/1512457027,v1:172.21.15.70:6819/1512457027] conn(0x7f3d80074c10 msgr2=0x7f3d80077030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.431+0000 7f3d86ffd640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.70:6817/1512457027,v1:172.21.15.70:6819/1512457027] conn(0x7f3d80074c10 0x7f3d80077030 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f3d86ffd640 1 -- 172.21.15.44:0/525652193 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3d8006c9f0 msgr2=0x7f3d8006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f3d86ffd640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f3d8006c9f0 0x7f3d8006eeb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400ef50 tx=0x7f3d9c06e030 comp rx=0 tx=0).stop 2024-04-22T07:04:21.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f3d86ffd640 1 -- 172.21.15.44:0/525652193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d940dc390 msgr2=0x7f3d940df030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f3d86ffd640 1 --2- 172.21.15.44:0/525652193 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3d940dc390 0x7f3d940df030 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000e7f0 tx=0x7f3d9000ecb0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f3da1600640 1 -- 172.21.15.44:0/525652193 reap_dead start 2024-04-22T07:04:21.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f13dccb3640 1 -- 172.21.15.44:0/2258115876 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13c406c970 msgr2=0x7f13c406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f13dccb3640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f13c406c970 0x7f13c406ee30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005880 tx=0x7f13c8009040 comp rx=0 tx=0).stop 2024-04-22T07:04:21.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f13dccb3640 1 -- 172.21.15.44:0/2258115876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d8154ba0 msgr2=0x7f13d8072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f13dccb3640 1 --2- 172.21.15.44:0/2258115876 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f13d8154ba0 0x7f13d8072570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f13d80d13f0 tx=0x7f13d0004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:21.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f13d7fff640 1 -- 172.21.15.44:0/2258115876 reap_dead start 2024-04-22T07:04:21.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f3d86ffd640 1 -- 172.21.15.44:0/525652193 shutdown_connections 2024-04-22T07:04:21.486 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f3d86ffd640 1 -- 172.21.15.44:0/525652193 >> 172.21.15.44:0/525652193 conn(0x7f3d94051290 msgr2=0x7f3d940d5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f663b7fe640 1 -- 172.21.15.44:0/4111515487 reap_dead start 2024-04-22T07:04:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 shutdown_connections 2024-04-22T07:04:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f13dccb3640 1 -- 172.21.15.44:0/2258115876 shutdown_connections 2024-04-22T07:04:21.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 >> 172.21.15.44:0/4111515487 conn(0x7f66340505d0 msgr2=0x7f663404ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f13dccb3640 1 -- 172.21.15.44:0/2258115876 >> 172.21.15.44:0/2258115876 conn(0x7f13d8073470 msgr2=0x7f13d815be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f3d86ffd640 1 -- 172.21.15.44:0/525652193 shutdown_connections 2024-04-22T07:04:21.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 shutdown_connections 2024-04-22T07:04:21.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.432+0000 7f3d86ffd640 1 -- 172.21.15.44:0/525652193 wait complete. 2024-04-22T07:04:21.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.433+0000 7f13dccb3640 1 -- 172.21.15.44:0/2258115876 shutdown_connections 2024-04-22T07:04:21.489 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.433+0000 7f13dccb3640 1 -- 172.21.15.44:0/2258115876 wait complete. 2024-04-22T07:04:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.433+0000 7f663bfff640 1 -- 172.21.15.44:0/4111515487 wait complete. 2024-04-22T07:04:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.433+0000 7f537d481640 1 -- 172.21.15.44:0/3075125521 --> [v2:172.21.15.44:6808/2837839457,v1:172.21.15.44:6809/2837839457] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f537804fce0 con 0x7f5364074c70 2024-04-22T07:04:21.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.434+0000 7f535f7fe640 1 -- 172.21.15.44:0/3075125521 <== osd.1 v2:172.21.15.44:6808/2837839457 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f537804fce0 con 0x7f5364074c70 2024-04-22T07:04:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.434+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 --> [v2:172.21.15.44:6816/2173398368,v1:172.21.15.44:6817/2173398368] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f138c002d50 con 0x7f1388074bc0 2024-04-22T07:04:21.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.435+0000 7f535d7fa640 1 -- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:6808/2837839457,v1:172.21.15.44:6809/2837839457] conn(0x7f5364074c70 msgr2=0x7f5364077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.435+0000 7f535d7fa640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:6808/2837839457,v1:172.21.15.44:6809/2837839457] conn(0x7f5364074c70 0x7f5364077090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.435+0000 7f535d7fa640 1 -- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f536406c9f0 msgr2=0x7f536406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.435+0000 7f535d7fa640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f536406c9f0 0x7f536406eeb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f536c036000 tx=0x7f536c008030 comp rx=0 tx=0).stop 2024-04-22T07:04:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.435+0000 7f535d7fa640 1 -- 172.21.15.44:0/3075125521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53780c3480 msgr2=0x7f5378157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.435+0000 7f535d7fa640 1 --2- 172.21.15.44:0/3075125521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f53780c3480 0x7f5378157f80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f536800d620 tx=0x7f536800dae0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.435+0000 7f53777fe640 1 -- 172.21.15.44:0/3075125521 reap_dead start 2024-04-22T07:04:21.492 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.435+0000 7f1396ffd640 1 -- 172.21.15.44:0/3836638405 <== osd.3 v2:172.21.15.44:6816/2173398368 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f138c002d50 con 0x7f1388074bc0 2024-04-22T07:04:21.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.436+0000 7f535d7fa640 1 -- 172.21.15.44:0/3075125521 shutdown_connections 2024-04-22T07:04:21.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.436+0000 7f535d7fa640 1 -- 172.21.15.44:0/3075125521 >> 172.21.15.44:0/3075125521 conn(0x7f5378071300 msgr2=0x7f53780c8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.436+0000 7f535d7fa640 1 -- 172.21.15.44:0/3075125521 shutdown_connections 2024-04-22T07:04:21.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.436+0000 7f535d7fa640 1 -- 172.21.15.44:0/3075125521 wait complete. 2024-04-22T07:04:21.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.436+0000 7f33a67fc640 1 -- 172.21.15.44:0/3846063616 --> [v2:172.21.15.70:6800/921795404,v1:172.21.15.70:6801/921795404] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f33b404bc60 con 0x7f33a0074be0 2024-04-22T07:04:21.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.436+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:6816/2173398368,v1:172.21.15.44:6817/2173398368] conn(0x7f1388074bc0 msgr2=0x7f1388076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.436+0000 7f13acc74640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:6816/2173398368,v1:172.21.15.44:6817/2173398368] conn(0x7f1388074bc0 0x7f1388076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.437+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f138806c940 msgr2=0x7f138806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.437+0000 7f13acc74640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f138806c940 0x7f138806ee00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f139c000f60 tx=0x7f139c0079b0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.437+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13a00e3090 msgr2=0x7f13a0179570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.437+0000 7f13acc74640 1 --2- 172.21.15.44:0/3836638405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13a00e3090 0x7f13a0179570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f139800e870 tx=0x7f139800ed30 comp rx=0 tx=0).stop 2024-04-22T07:04:21.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.437+0000 7f13a6d9d640 1 -- 172.21.15.44:0/3836638405 reap_dead start 2024-04-22T07:04:21.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.437+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 shutdown_connections 2024-04-22T07:04:21.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.437+0000 7f33b8ff9640 1 -- 172.21.15.44:0/3846063616 <== osd.6 v2:172.21.15.70:6800/921795404 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f33b404bc60 con 0x7f33a0074be0 2024-04-22T07:04:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.437+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 >> 172.21.15.44:0/3836638405 conn(0x7f13a0050180 msgr2=0x7f13a00e13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.437+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 shutdown_connections 2024-04-22T07:04:21.496 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.437+0000 7f13acc74640 1 -- 172.21.15.44:0/3836638405 wait complete. 2024-04-22T07:04:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.438+0000 7f33a67fc640 1 -- 172.21.15.44:0/3846063616 >> [v2:172.21.15.70:6800/921795404,v1:172.21.15.70:6801/921795404] conn(0x7f33a0074be0 msgr2=0x7f33a0077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.438+0000 7f33a67fc640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.70:6800/921795404,v1:172.21.15.70:6801/921795404] conn(0x7f33a0074be0 0x7f33a0077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.497 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.438+0000 7f33a67fc640 1 -- 172.21.15.44:0/3846063616 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c9c0 msgr2=0x7f33a006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.438+0000 7f33a67fc640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c9c0 0x7f33a006ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33b0036000 tx=0x7f33b0008030 comp rx=0 tx=0).stop 2024-04-22T07:04:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.438+0000 7f33a67fc640 1 -- 172.21.15.44:0/3846063616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b404b090 msgr2=0x7f33b40cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.438+0000 7f33a67fc640 1 --2- 172.21.15.44:0/3846063616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b404b090 0x7f33b40cf160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f33a800c4d0 tx=0x7f33a800c990 comp rx=0 tx=0).stop 2024-04-22T07:04:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.438+0000 7f33c0df9640 1 -- 172.21.15.44:0/3846063616 reap_dead start 2024-04-22T07:04:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.438+0000 7f33a67fc640 1 -- 172.21.15.44:0/3846063616 shutdown_connections 2024-04-22T07:04:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.438+0000 7f33a67fc640 1 -- 172.21.15.44:0/3846063616 >> 172.21.15.44:0/3846063616 conn(0x7f33b4050180 msgr2=0x7f33b40cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.439+0000 7f33a67fc640 1 -- 172.21.15.44:0/3846063616 shutdown_connections 2024-04-22T07:04:21.498 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.439+0000 7f33a67fc640 1 -- 172.21.15.44:0/3846063616 wait complete. 2024-04-22T07:04:21.499 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-04-22T07:04:21.500 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-22T07:04:21.500 INFO:teuthology.orchestra.run.smithi044.stdout:55834574856 2024-04-22T07:04:21.501 INFO:teuthology.orchestra.run.smithi044.stdout:64424509449 2024-04-22T07:04:21.502 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-22T07:04:21.502 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-22T07:04:21.503 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-04-22T07:04:21.503 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-22T07:04:21.504 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-04-22T07:04:21.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-22T07:04:21.505 INFO:teuthology.orchestra.run.smithi044.stdout:64424509449 2024-04-22T07:04:21.505 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T07:04:21.506 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-04-22T07:04:21.506 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-22T07:04:21.507 INFO:teuthology.orchestra.run.smithi044.stdout:64424509449 2024-04-22T07:04:21.507 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-22T07:04:21.714 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.715+0000 7f1b29585640 1 Processor -- start 2024-04-22T07:04:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.715+0000 7f1b29585640 1 -- start start 2024-04-22T07:04:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.716+0000 7f1b29585640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f1b24071620 0x7f1b2415ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.716+0000 7f1b29585640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24072230 0x7f1b24072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.716+0000 7f1b29585640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b24156f10 con 0x7f1b24150ed0 2024-04-22T07:04:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.716+0000 7f1b29585640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f1b24071d40 con 0x7f1b24071620 2024-04-22T07:04:21.716 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.716+0000 7f1b29585640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2415b1c0 con 0x7f1b24072230 2024-04-22T07:04:21.717 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.717+0000 7f1b237fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24072230 0x7f1b24072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.717+0000 7f1b237fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24072230 0x7f1b24072680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:59054/0 (socket says 172.21.15.44:59054) 2024-04-22T07:04:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.717+0000 7f1b237fe640 1 -- 172.21.15.44:0/3232913369 learned_addr learned my addr 172.21.15.44:0/3232913369 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.717+0000 7f1b227fc640 1 --2- 172.21.15.44:0/3232913369 >> v2:172.21.15.44:3300/0 conn(0x7f1b24071620 0x7f1b2415ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.718+0000 7f1b227fc640 1 -- 172.21.15.44:0/3232913369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24072230 msgr2=0x7f1b24072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.718+0000 7f1b227fc640 1 --2- 172.21.15.44:0/3232913369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24072230 0x7f1b24072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.718+0000 7f1b227fc640 1 -- 172.21.15.44:0/3232913369 >> v1:172.21.15.70:6789/0 conn(0x7f1b24150ed0 legacy=0x7f1b2406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.718 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.718+0000 7f1b227fc640 1 -- 172.21.15.44:0/3232913369 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b24072bc0 con 0x7f1b24071620 2024-04-22T07:04:21.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.719+0000 7f6de359e640 1 Processor -- start 2024-04-22T07:04:21.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.719+0000 7f6de359e640 1 -- start start 2024-04-22T07:04:21.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.718+0000 7fe815281640 1 Processor -- start 2024-04-22T07:04:21.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.718+0000 7f0691bf4640 1 Processor -- start 2024-04-22T07:04:21.719 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.719+0000 7f0691bf4640 1 -- start start 2024-04-22T07:04:21.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.719+0000 7ff13e67e640 1 Processor -- start 2024-04-22T07:04:21.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.718+0000 7fe815281640 1 -- start start 2024-04-22T07:04:21.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.719+0000 7fe815281640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe8100c8d90 0x7fe81015f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7fe815281640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe81006cd00 0x7fe8101696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7fe815281640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe810073270 con 0x7fe8100c9170 2024-04-22T07:04:21.720 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7fe815281640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fe81015efb0 con 0x7fe8100c8d90 2024-04-22T07:04:21.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7fe815281640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe8100c6260 con 0x7fe81006cd00 2024-04-22T07:04:21.721 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7ff13e67e640 1 -- start start 2024-04-22T07:04:21.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7f0691bf4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f068404ba50 0x7f068404be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7f0691bf4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06840e3080 0x7f06840e34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7f0691bf4640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0684046b60 con 0x7f06840dd3b0 2024-04-22T07:04:21.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7f0691bf4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f0684045440 con 0x7f068404ba50 2024-04-22T07:04:21.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7f0691bf4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0684045fc0 con 0x7f06840e3080 2024-04-22T07:04:21.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7f068affd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f068404ba50 0x7f068404be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7f068affd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f068404ba50 0x7f068404be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38038/0 (socket says 172.21.15.44:38038) 2024-04-22T07:04:21.722 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7f068affd640 1 -- 172.21.15.44:0/1834577563 learned_addr learned my addr 172.21.15.44:0/1834577563 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f1b227fc640 1 --2- 172.21.15.44:0/3232913369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24071620 0x7f1b2415ac80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009a00 tx=0x7f1b1802bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca4ca96bb5157e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f6de359e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6de4158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f6de359e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6de415f380 con 0x7f6de4072b60 2024-04-22T07:04:21.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f6de359e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6de415f660 con 0x7f6de406e3e0 2024-04-22T07:04:21.723 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f6de359e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6de415f940 con 0x7f6de40c7c30 2024-04-22T07:04:21.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f6de259c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f6de4072b60 0x7f6de4072f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38078/0 (socket says 172.21.15.44:38078) 2024-04-22T07:04:21.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f6de259c640 1 -- 172.21.15.44:0/1541341475 learned_addr learned my addr 172.21.15.44:0/1541341475 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:21.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7ff13e67e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff1300dd4c0 0x7ff13004e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7ff13e67e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1300e3440 con 0x7ff13004e6d0 2024-04-22T07:04:21.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7ff13e67e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1300e3720 con 0x7ff1300e6c20 2024-04-22T07:04:21.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7ff13e67e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff130049510 con 0x7ff1300dd4c0 2024-04-22T07:04:21.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7fe80effd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe8100c8d90 0x7fe81015f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7fe80effd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe8100c8d90 0x7fe81015f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38022/0 (socket says 172.21.15.44:38022) 2024-04-22T07:04:21.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.720+0000 7fe80effd640 1 -- 172.21.15.44:0/3485449195 learned_addr learned my addr 172.21.15.44:0/3485449195 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7fe80effd640 1 -- 172.21.15.44:0/3485449195 >> v2:172.21.15.70:3300/0 conn(0x7fe81006cd00 msgr2=0x7fe8101696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7fe80effd640 1 --2- 172.21.15.44:0/3485449195 >> v2:172.21.15.70:3300/0 conn(0x7fe81006cd00 0x7fe8101696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7fe80effd640 1 -- 172.21.15.44:0/3485449195 >> v1:172.21.15.44:6789/0 conn(0x7fe8100c9170 legacy=0x7fe810158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.725 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7fe80effd640 1 -- 172.21.15.44:0/3485449195 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe810072f90 con 0x7fe8100c8d90 2024-04-22T07:04:21.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7ff13de7d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff1300e6c20 0x7ff1300e7060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:51734/0 (socket says 172.21.15.44:51734) 2024-04-22T07:04:21.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7ff13de7d640 1 -- 172.21.15.44:0/82021938 learned_addr learned my addr 172.21.15.44:0/82021938 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:21.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7ff13d67c640 1 --2- 172.21.15.44:0/82021938 >> v2:172.21.15.70:3300/0 conn(0x7ff1300dd4c0 0x7ff13004e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118579726 0 0) 0x7f6de415f660 con 0x7f6de406e3e0 2024-04-22T07:04:21.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f068affd640 1 -- 172.21.15.44:0/1834577563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06840e3080 msgr2=0x7f06840e34f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:04:21.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f068affd640 1 --2- 172.21.15.44:0/1834577563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06840e3080 0x7f06840e34f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f068affd640 1 -- 172.21.15.44:0/1834577563 >> v1:172.21.15.44:6789/0 conn(0x7f06840dd3b0 legacy=0x7f0684046450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.721+0000 7f068affd640 1 -- 172.21.15.44:0/1834577563 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06840dd790 con 0x7f068404ba50 2024-04-22T07:04:21.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dcc003520 con 0x7f6de406e3e0 2024-04-22T07:04:21.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7fb6e0a43640 1 Processor -- start 2024-04-22T07:04:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7ff13d67c640 1 -- 172.21.15.44:0/82021938 >> v1:172.21.15.70:6789/0 conn(0x7ff1300e6c20 legacy=0x7ff1300e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7ff13d67c640 1 -- 172.21.15.44:0/82021938 >> v1:172.21.15.44:6789/0 conn(0x7ff13004e6d0 legacy=0x7ff13004edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7ff13d67c640 1 -- 172.21.15.44:0/82021938 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1300e3a00 con 0x7ff1300dd4c0 2024-04-22T07:04:21.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 <== mon.1 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3898557258 0 0) 0x7f6dcc003520 con 0x7f6de406e3e0 2024-04-22T07:04:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7f1b21ffb640 1 -- 172.21.15.44:0/3232913369 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1802d6e0 con 0x7f1b24071620 2024-04-22T07:04:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7f1b21ffb640 1 -- 172.21.15.44:0/3232913369 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1802d880 con 0x7f1b24071620 2024-04-22T07:04:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7f1b21ffb640 1 -- 172.21.15.44:0/3232913369 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1802dc90 con 0x7f1b24071620 2024-04-22T07:04:21.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 msgr2=0x7f6de4158cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T07:04:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7f6de159a640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6de4158cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 >> v1:172.21.15.44:6789/0 conn(0x7f6de4072b60 legacy=0x7f6de4072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de415fc20 con 0x7f6de406e3e0 2024-04-22T07:04:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7feafdea2640 1 Processor -- start 2024-04-22T07:04:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7feafdea2640 1 -- start start 2024-04-22T07:04:21.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 <== mon.1 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3718881788 0 0) 0x7f6dd8001670 con 0x7f6de406e3e0 2024-04-22T07:04:21.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f068affd640 1 --2- 172.21.15.44:0/1834577563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f068404ba50 0x7f068404be70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f068c065320 tx=0x7f068c0a7ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0212be3c30ecbb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.731 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7fe80effd640 1 --2- 172.21.15.44:0/3485449195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8100c8d90 0x7fe81015f770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe808009e50 tx=0x7fe80802c020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0e78162da45497 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 >> v1:172.21.15.70:6789/0 conn(0x7f6de406e3e0 legacy=0x7f6de40c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.722+0000 7fb6e0a43640 1 -- start start 2024-04-22T07:04:21.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de159a640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dcc0039b0 0x7f6dcc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de159a640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6dcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de159a640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dcc004db0 0x7f6dcc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6de415f940 con 0x7f6dcc004db0 2024-04-22T07:04:21.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7fe80dffb640 1 -- 172.21.15.44:0/3485449195 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80802cbc0 con 0x7fe8100c8d90 2024-04-22T07:04:21.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7fe80dffb640 1 -- 172.21.15.44:0/3485449195 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe80802cd60 con 0x7fe8100c8d90 2024-04-22T07:04:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc00bc90 con 0x7f6dcc0039b0 2024-04-22T07:04:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7ff13d67c640 1 --2- 172.21.15.44:0/82021938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1300dd4c0 0x7ff13004e090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff128009720 tx=0x7ff12802b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc04df45d7ad30e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6dcc00be90 con 0x7f6de40c7c30 2024-04-22T07:04:21.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7fe80dffb640 1 -- 172.21.15.44:0/3485449195 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80802cbc0 con 0x7fe8100c8d90 2024-04-22T07:04:21.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7feafdea2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feaf8159150 0x7feaf8162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7feafdea2640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7feaf80c7080 0x7feaf80676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7feafdea2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c74a0 0x7feaf8067bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de259c640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dcc0039b0 0x7f6dcc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de1d9b640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6dcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.736 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f1b29585640 1 -- 172.21.15.44:0/3232913369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24071620 msgr2=0x7f1b2415ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f1b29585640 1 --2- 172.21.15.44:0/3232913369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24071620 0x7f1b2415ac80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009a00 tx=0x7f1b1802bb30 comp rx=0 tx=0).stop 2024-04-22T07:04:21.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f6de2d9d640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dcc004db0 0x7f6dcc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.737 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7f1b29585640 1 -- 172.21.15.44:0/3232913369 shutdown_connections 2024-04-22T07:04:21.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafdea2640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7feaf8162850 con 0x7feaf80c7080 2024-04-22T07:04:21.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafdea2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7feaf806bd50 con 0x7feaf8159150 2024-04-22T07:04:21.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafdea2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7feaf806c030 con 0x7feaf80c74a0 2024-04-22T07:04:21.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7f1b29585640 1 --2- 172.21.15.44:0/3232913369 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24072230 0x7f1b24072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.738 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7f1b29585640 1 --2- 172.21.15.44:0/3232913369 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24071620 0x7f1b2415ac80 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1b18009a00 tx=0x7f1b1802bb30 comp rx=0 tx=0).stop 2024-04-22T07:04:21.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7f1b29585640 1 -- 172.21.15.44:0/3232913369 >> 172.21.15.44:0/3232913369 conn(0x7f1b240cc020 msgr2=0x7f1b240cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f068a7fc640 1 -- 172.21.15.44:0/1834577563 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c0a9950 con 0x7f068404ba50 2024-04-22T07:04:21.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.723+0000 7f068a7fc640 1 -- 172.21.15.44:0/1834577563 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068c0a9af0 con 0x7f068404ba50 2024-04-22T07:04:21.739 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7f068a7fc640 1 -- 172.21.15.44:0/1834577563 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c0aa460 con 0x7f068404ba50 2024-04-22T07:04:21.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6e0a43640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc151c70 0x7fb6dc157810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6e0a43640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb6dc0c4780 0x7fb6dc0cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6e0a43640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6dc0c8370 con 0x7fb6dc06cf60 2024-04-22T07:04:21.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6e0a43640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb6dc0c6440 con 0x7fb6dc0c4780 2024-04-22T07:04:21.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6e0a43640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc0c6720 con 0x7fb6dc151c70 2024-04-22T07:04:21.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafd6a1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c74a0 0x7feaf8067bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafd6a1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c74a0 0x7feaf8067bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:59090/0 (socket says 172.21.15.44:59090) 2024-04-22T07:04:21.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafd6a1640 1 -- 172.21.15.44:0/3076660352 learned_addr learned my addr 172.21.15.44:0/3076660352 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feaf7fff640 1 --2- 172.21.15.44:0/3076660352 >> v2:172.21.15.44:3300/0 conn(0x7feaf80c7080 0x7feaf80676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafcea0640 1 --2- 172.21.15.44:0/3076660352 >> v2:172.21.15.70:3300/0 conn(0x7feaf8159150 0x7feaf8162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6d9d9b640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb6dc06cf60 0x7fb6dc0c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38098/0 (socket says 172.21.15.44:38098) 2024-04-22T07:04:21.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6d9d9b640 1 -- 172.21.15.44:0/3689841509 learned_addr learned my addr 172.21.15.44:0/3689841509 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:21.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6dad9d640 1 --2- 172.21.15.44:0/3689841509 >> v2:172.21.15.70:3300/0 conn(0x7fb6dc0c4780 0x7fb6dc0cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7f1b29585640 1 -- 172.21.15.44:0/3232913369 shutdown_connections 2024-04-22T07:04:21.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7ff12ffff640 1 -- 172.21.15.44:0/82021938 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128037070 con 0x7ff1300dd4c0 2024-04-22T07:04:21.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7ff12ffff640 1 -- 172.21.15.44:0/82021938 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff128030710 con 0x7ff1300dd4c0 2024-04-22T07:04:21.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafd6a1640 1 -- 172.21.15.44:0/3076660352 >> v2:172.21.15.70:3300/0 conn(0x7feaf8159150 msgr2=0x7feaf8162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafd6a1640 1 --2- 172.21.15.44:0/3076660352 >> v2:172.21.15.70:3300/0 conn(0x7feaf8159150 0x7feaf8162ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafd6a1640 1 -- 172.21.15.44:0/3076660352 >> v2:172.21.15.44:3300/0 conn(0x7feaf80c7080 msgr2=0x7feaf80676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafd6a1640 1 --2- 172.21.15.44:0/3076660352 >> v2:172.21.15.44:3300/0 conn(0x7feaf80c7080 0x7feaf80676b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7feafd6a1640 1 -- 172.21.15.44:0/3076660352 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf806c310 con 0x7feaf80c74a0 2024-04-22T07:04:21.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6da59c640 1 --2- 172.21.15.44:0/3689841509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc151c70 0x7fb6dc157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7f1b29585640 1 -- 172.21.15.44:0/3232913369 wait complete. 2024-04-22T07:04:21.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f0691bf4640 1 -- 172.21.15.44:0/1834577563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f068404ba50 msgr2=0x7f068404be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.744 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f0691bf4640 1 --2- 172.21.15.44:0/1834577563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f068404ba50 0x7f068404be70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f068c065320 tx=0x7f068c0a7ad0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.745 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f6de1d9b640 1 -- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dcc0039b0 msgr2=0x7f6dcc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fe815281640 1 -- 172.21.15.44:0/3485449195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8100c8d90 msgr2=0x7fe81015f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fe815281640 1 --2- 172.21.15.44:0/3485449195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8100c8d90 0x7fe81015f770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe808009e50 tx=0x7fe80802c020 comp rx=0 tx=0).stop 2024-04-22T07:04:21.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f0691bf4640 1 -- 172.21.15.44:0/1834577563 shutdown_connections 2024-04-22T07:04:21.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f0691bf4640 1 --2- 172.21.15.44:0/1834577563 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06840e3080 0x7f06840e34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f0691bf4640 1 --2- 172.21.15.44:0/1834577563 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f068404ba50 0x7f068404be70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f0691bf4640 1 -- 172.21.15.44:0/1834577563 >> 172.21.15.44:0/1834577563 conn(0x7f0684050170 msgr2=0x7f0684050580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f6de1d9b640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dcc0039b0 0x7f6dcc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f6de1d9b640 1 -- 172.21.15.44:0/1541341475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dcc004db0 msgr2=0x7f6dcc005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f6de1d9b640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6dcc004db0 0x7f6dcc005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.747 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f6de1d9b640 1 -- 172.21.15.44:0/1541341475 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de415fc20 con 0x7f6de40c7c30 2024-04-22T07:04:21.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6dad9d640 1 -- 172.21.15.44:0/3689841509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc151c70 msgr2=0x7fb6dc157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6dad9d640 1 --2- 172.21.15.44:0/3689841509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc151c70 0x7fb6dc157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6dad9d640 1 -- 172.21.15.44:0/3689841509 >> v1:172.21.15.44:6789/0 conn(0x7fb6dc06cf60 legacy=0x7fb6dc0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.724+0000 7fb6dad9d640 1 -- 172.21.15.44:0/3689841509 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc0c6a00 con 0x7fb6dc0c4780 2024-04-22T07:04:21.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7fe815281640 1 -- 172.21.15.44:0/3485449195 shutdown_connections 2024-04-22T07:04:21.748 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7fe815281640 1 --2- 172.21.15.44:0/3485449195 >> v2:172.21.15.70:3300/0 conn(0x7fe81006cd00 0x7fe8101696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7fe815281640 1 --2- 172.21.15.44:0/3485449195 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe8100c8d90 0x7fe81015f770 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe808009e50 tx=0x7fe80802c020 comp rx=0 tx=0).stop 2024-04-22T07:04:21.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7fe815281640 1 -- 172.21.15.44:0/3485449195 >> 172.21.15.44:0/3485449195 conn(0x7fe81006b970 msgr2=0x7fe81006bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7ff13e67e640 1 -- 172.21.15.44:0/82021938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1300dd4c0 msgr2=0x7ff13004e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7ff13e67e640 1 --2- 172.21.15.44:0/82021938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1300dd4c0 0x7ff13004e090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff128009720 tx=0x7ff12802b710 comp rx=0 tx=0).stop 2024-04-22T07:04:21.749 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7fe815281640 1 -- 172.21.15.44:0/3485449195 shutdown_connections 2024-04-22T07:04:21.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f0691bf4640 1 -- 172.21.15.44:0/1834577563 shutdown_connections 2024-04-22T07:04:21.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7f0691bf4640 1 -- 172.21.15.44:0/1834577563 wait complete. 2024-04-22T07:04:21.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7feafd6a1640 1 --2- 172.21.15.44:0/3076660352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c74a0 0x7feaf8067bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feaf8073aa0 tx=0x7feaf002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a798bf909c310a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7fb6dad9d640 1 --2- 172.21.15.44:0/3689841509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc0c4780 0x7fb6dc0cd1d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400bfa0 tx=0x7fb6d402e150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfd0faa182ac7cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.750 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7feaf77fe640 1 -- 172.21.15.44:0/3076660352 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf002cb00 con 0x7feaf80c74a0 2024-04-22T07:04:21.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7feaf77fe640 1 -- 172.21.15.44:0/3076660352 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf002cca0 con 0x7feaf80c74a0 2024-04-22T07:04:21.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7feaf77fe640 1 -- 172.21.15.44:0/3076660352 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0034610 con 0x7feaf80c74a0 2024-04-22T07:04:21.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.725+0000 7fe815281640 1 -- 172.21.15.44:0/3485449195 wait complete. 2024-04-22T07:04:21.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7fb6d959a640 1 -- 172.21.15.44:0/3689841509 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d402eaf0 con 0x7fb6dc0c4780 2024-04-22T07:04:21.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7fb6d959a640 1 -- 172.21.15.44:0/3689841509 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d402ec90 con 0x7fb6dc0c4780 2024-04-22T07:04:21.751 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7fb6d959a640 1 -- 172.21.15.44:0/3689841509 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4009760 con 0x7fb6dc0c4780 2024-04-22T07:04:21.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7ff13e67e640 1 -- 172.21.15.44:0/82021938 shutdown_connections 2024-04-22T07:04:21.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7f1b29585640 1 Processor -- start 2024-04-22T07:04:21.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7f6de1d9b640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6dcc004800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c1e0 tx=0x7f6dcc005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1e06a2b216d8db server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7ff13e67e640 1 --2- 172.21.15.44:0/82021938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1300dd4c0 0x7ff13004e090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7ff13e67e640 1 -- 172.21.15.44:0/82021938 >> 172.21.15.44:0/82021938 conn(0x7ff13000e6a0 msgr2=0x7ff130009da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8039070 con 0x7f6de40c7c30 2024-04-22T07:04:21.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3689841509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc0c4780 msgr2=0x7fb6dc0cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7fb6e0a43640 1 --2- 172.21.15.44:0/3689841509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc0c4780 0x7fb6dc0cd1d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400bfa0 tx=0x7fb6d402e150 comp rx=0 tx=0).stop 2024-04-22T07:04:21.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7f1b29585640 1 -- start start 2024-04-22T07:04:21.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f0691bf4640 1 Processor -- start 2024-04-22T07:04:21.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.726+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd802ca10 con 0x7f6de40c7c30 2024-04-22T07:04:21.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f6de359e640 1 -- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 msgr2=0x7f6dcc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f6de159a640 1 -- 172.21.15.44:0/1541341475 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8039070 con 0x7f6de40c7c30 2024-04-22T07:04:21.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7ff13e67e640 1 -- 172.21.15.44:0/82021938 shutdown_connections 2024-04-22T07:04:21.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7feafdea2640 1 -- 172.21.15.44:0/3076660352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c74a0 msgr2=0x7feaf8067bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7feafdea2640 1 --2- 172.21.15.44:0/3076660352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c74a0 0x7feaf8067bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feaf8073aa0 tx=0x7feaf002bf60 comp rx=0 tx=0).stop 2024-04-22T07:04:21.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3689841509 shutdown_connections 2024-04-22T07:04:21.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7fb6e0a43640 1 --2- 172.21.15.44:0/3689841509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc0c4780 0x7fb6dc0cd1d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7fb6e0a43640 1 --2- 172.21.15.44:0/3689841509 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc151c70 0x7fb6dc157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.755 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3689841509 >> 172.21.15.44:0/3689841509 conn(0x7fb6dc072690 msgr2=0x7fb6dc072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f1b29585640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24072230 0x7f1b240be040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f1b29585640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24150ed0 0x7f1b240c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f1b29585640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b240be630 0x7f1b241f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f1b29585640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1b241571b0 con 0x7f1b24072230 2024-04-22T07:04:21.756 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f1b29585640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1b24071a70 con 0x7f1b240be630 2024-04-22T07:04:21.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f1b29585640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1b24157490 con 0x7f1b24150ed0 2024-04-22T07:04:21.757 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f6de359e640 1 --2- 172.21.15.44:0/1541341475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6dcc004800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c1e0 tx=0x7f6dcc005830 comp rx=0 tx=0).stop 2024-04-22T07:04:21.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f384308f640 1 Processor -- start 2024-04-22T07:04:21.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f6de2d9d640 1 -- 172.21.15.44:0/1541341475 reap_dead start 2024-04-22T07:04:21.758 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7ff13e67e640 1 -- 172.21.15.44:0/82021938 wait complete. 2024-04-22T07:04:21.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f1b22ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24072230 0x7f1b240be040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f1b22ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24072230 0x7f1b240be040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38048/0 (socket says 172.21.15.44:38048) 2024-04-22T07:04:21.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f1b22ffd640 1 -- 172.21.15.44:0/3794673975 learned_addr learned my addr 172.21.15.44:0/3794673975 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f6de359e640 1 -- 172.21.15.44:0/1541341475 shutdown_connections 2024-04-22T07:04:21.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f1b227fc640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24150ed0 0x7f1b240c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.759 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f1b237fe640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b240be630 0x7f1b241f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7fe815281640 1 Processor -- start 2024-04-22T07:04:21.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7fe815281640 1 -- start start 2024-04-22T07:04:21.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f6de359e640 1 -- 172.21.15.44:0/1541341475 >> 172.21.15.44:0/1541341475 conn(0x7f6de40cc8b0 msgr2=0x7f6de40cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.760 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3689841509 shutdown_connections 2024-04-22T07:04:21.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f384308f640 1 -- start start 2024-04-22T07:04:21.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.727+0000 7f0691bf4640 1 -- start start 2024-04-22T07:04:21.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f1b22ffd640 1 -- 172.21.15.44:0/3794673975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24150ed0 msgr2=0x7f1b240c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f1b22ffd640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24150ed0 0x7f1b240c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f1b22ffd640 1 -- 172.21.15.44:0/3794673975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b240be630 msgr2=0x7f1b241f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f1b22ffd640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b240be630 0x7f1b241f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7fe815281640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe81006cd00 0x7fe8100cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7f1b22ffd640 1 -- 172.21.15.44:0/3794673975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b241f10f0 con 0x7f1b24072230 2024-04-22T07:04:21.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7fe815281640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8100c9170 0x7fe81015e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f1b22ffd640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24072230 0x7f1b240be040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00bab0 tx=0x7f1b1c00bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7fe815281640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81013eea0 0x7fe81015e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7feafdea2640 1 -- 172.21.15.44:0/3076660352 shutdown_connections 2024-04-22T07:04:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7feafdea2640 1 --2- 172.21.15.44:0/3076660352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c74a0 0x7feaf8067bf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7feafdea2640 1 --2- 172.21.15.44:0/3076660352 >> v2:172.21.15.44:3300/0 conn(0x7feaf80c7080 0x7feaf80676b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7feafdea2640 1 --2- 172.21.15.44:0/3076660352 >> v2:172.21.15.70:3300/0 conn(0x7feaf8159150 0x7feaf8162ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7feafdea2640 1 -- 172.21.15.44:0/3076660352 >> 172.21.15.44:0/3076660352 conn(0x7feaf80c9db0 msgr2=0x7feaf80ca190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7fe815281640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe810071690 con 0x7fe81013eea0 2024-04-22T07:04:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7fe815281640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe81015fcb0 con 0x7fe8100c9170 2024-04-22T07:04:21.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7fe815281640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe810071810 con 0x7fe81006cd00 2024-04-22T07:04:21.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f0691bf4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f068404ba50 0x7f06840cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f0691bf4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06840dd3b0 0x7f06840dbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f0691bf4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06840e3080 0x7f0684179630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f0691bf4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f0684045970 con 0x7f06840dd3b0 2024-04-22T07:04:21.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f0691bf4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0684045690 con 0x7f068404ba50 2024-04-22T07:04:21.765 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f0691bf4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f0684045c50 con 0x7f06840e3080 2024-04-22T07:04:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7fe80e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8100c9170 0x7fe81015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7fe80ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81013eea0 0x7fe81015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7fe80e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8100c9170 0x7fe81015e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36904/0 (socket says 172.21.15.44:36904) 2024-04-22T07:04:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7fe80ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81013eea0 0x7fe81015e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38062/0 (socket says 172.21.15.44:38062) 2024-04-22T07:04:21.766 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7fe80ffff640 1 -- 172.21.15.44:0/1392244496 learned_addr learned my addr 172.21.15.44:0/1392244496 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f068affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06840dd3b0 0x7f06840dbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f068b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f068404ba50 0x7f06840cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f068affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06840dd3b0 0x7f06840dbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38056/0 (socket says 172.21.15.44:38056) 2024-04-22T07:04:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f068affd640 1 -- 172.21.15.44:0/2777032692 learned_addr learned my addr 172.21.15.44:0/2777032692 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f068bfff640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06840e3080 0x7f0684179630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7ff13e67e640 1 Processor -- start 2024-04-22T07:04:21.767 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f6de359e640 1 -- 172.21.15.44:0/1541341475 shutdown_connections 2024-04-22T07:04:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7ff13e67e640 1 -- start start 2024-04-22T07:04:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f384308f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38340dd4d0 0x7f3834045560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f384308f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f3834045aa0 0x7f383404bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f384308f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38340dd8b0 con 0x7f383404c3e0 2024-04-22T07:04:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f384308f640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f38340e4050 con 0x7f3834045aa0 2024-04-22T07:04:21.768 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f384308f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f38340e34d0 con 0x7f38340dd4d0 2024-04-22T07:04:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fe80e7fc640 1 -- 172.21.15.44:0/1392244496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe81006cd00 msgr2=0x7fe8100cc6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T07:04:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fe80e7fc640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe81006cd00 0x7fe8100cc6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fe80e7fc640 1 -- 172.21.15.44:0/1392244496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81013eea0 msgr2=0x7fe81015e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fe80e7fc640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81013eea0 0x7fe81015e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.769 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fe80e7fc640 1 -- 172.21.15.44:0/1392244496 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe810141220 con 0x7fe8100c9170 2024-04-22T07:04:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f384208d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38340dd4d0 0x7f3834045560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f384208d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38340dd4d0 0x7f3834045560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:59136/0 (socket says 172.21.15.44:59136) 2024-04-22T07:04:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f384208d640 1 -- 172.21.15.44:0/3547998499 learned_addr learned my addr 172.21.15.44:0/3547998499 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.770 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f6de359e640 1 -- 172.21.15.44:0/1541341475 wait complete. 2024-04-22T07:04:21.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f068bfff640 1 -- 172.21.15.44:0/2777032692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f068404ba50 msgr2=0x7f06840cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f068bfff640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f068404ba50 0x7f06840cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f068bfff640 1 -- 172.21.15.44:0/2777032692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06840dd3b0 msgr2=0x7f06840dbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f068bfff640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06840dd3b0 0x7f06840dbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.771 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.729+0000 7f068bfff640 1 -- 172.21.15.44:0/2777032692 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06840dba00 con 0x7f06840e3080 2024-04-22T07:04:21.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f068bfff640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06840e3080 0x7f0684179630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f067800b410 tx=0x7f067800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f0677fff640 1 -- 172.21.15.44:0/2777032692 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678015020 con 0x7f06840e3080 2024-04-22T07:04:21.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f0677fff640 1 -- 172.21.15.44:0/2777032692 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0678011830 con 0x7f06840e3080 2024-04-22T07:04:21.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f0677fff640 1 -- 172.21.15.44:0/2777032692 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678011c20 con 0x7f06840e3080 2024-04-22T07:04:21.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f0691bf4640 1 -- 172.21.15.44:0/2777032692 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068417d000 con 0x7f06840e3080 2024-04-22T07:04:21.772 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f384188c640 1 --2- 172.21.15.44:0/3547998499 >> v2:172.21.15.44:3300/0 conn(0x7f3834045aa0 0x7f383404bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fe80e7fc640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8100c9170 0x7fe81015e310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe80400c3c0 tx=0x7fe80400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f0691bf4640 1 -- 172.21.15.44:0/2777032692 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0684178e30 con 0x7f06840e3080 2024-04-22T07:04:21.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f1b0b7fe640 1 -- 172.21.15.44:0/3794673975 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c00cc00 con 0x7f1b24072230 2024-04-22T07:04:21.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f1b0b7fe640 1 -- 172.21.15.44:0/3794673975 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1c00cdc0 con 0x7f1b24072230 2024-04-22T07:04:21.773 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7f1b0b7fe640 1 -- 172.21.15.44:0/3794673975 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c003bb0 con 0x7f1b24072230 2024-04-22T07:04:21.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fe7f77fe640 1 -- 172.21.15.44:0/1392244496 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804017070 con 0x7fe8100c9170 2024-04-22T07:04:21.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fe7f77fe640 1 -- 172.21.15.44:0/1392244496 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8040052d0 con 0x7fe8100c9170 2024-04-22T07:04:21.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fe7f77fe640 1 -- 172.21.15.44:0/1392244496 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804017070 con 0x7fe8100c9170 2024-04-22T07:04:21.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fe815281640 1 -- 172.21.15.44:0/1392244496 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe810153290 con 0x7fe8100c9170 2024-04-22T07:04:21.774 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13e67e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff13004e6d0 0x7ff1300ded80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f384188c640 1 -- 172.21.15.44:0/3547998499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38340dd4d0 msgr2=0x7f3834045560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:21.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7feafdea2640 1 -- 172.21.15.44:0/3076660352 shutdown_connections 2024-04-22T07:04:21.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f384188c640 1 --2- 172.21.15.44:0/3547998499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38340dd4d0 0x7f3834045560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f384188c640 1 -- 172.21.15.44:0/3547998499 >> v1:172.21.15.70:6789/0 conn(0x7f383404c3e0 legacy=0x7f3834046510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f6de359e640 1 Processor -- start 2024-04-22T07:04:21.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fe815281640 1 -- 172.21.15.44:0/1392244496 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe810153530 con 0x7fe8100c9170 2024-04-22T07:04:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13d67c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff13004e6d0 0x7ff1300ded80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13d67c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff13004e6d0 0x7ff1300ded80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38080/0 (socket says 172.21.15.44:38080) 2024-04-22T07:04:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13d67c640 1 -- 172.21.15.44:0/670252588 learned_addr learned my addr 172.21.15.44:0/670252588 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13e67e640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1300dd4c0 0x7ff13017ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13e67e640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1300e6c20 0x7ff1300e57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13ce7b640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1300dd4c0 0x7ff13017ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff1300e3fc0 con 0x7ff13004e6d0 2024-04-22T07:04:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff1300dd8a0 con 0x7ff1300e6c20 2024-04-22T07:04:21.777 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff1300e6700 con 0x7ff1300dd4c0 2024-04-22T07:04:21.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f0675ffb640 1 -- 172.21.15.44:0/2777032692 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0664005290 con 0x7f06840e3080 2024-04-22T07:04:21.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f384188c640 1 -- 172.21.15.44:0/3547998499 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38340e37b0 con 0x7f3834045aa0 2024-04-22T07:04:21.778 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f1b29585640 1 -- 172.21.15.44:0/3794673975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b241f13b0 con 0x7f1b24072230 2024-04-22T07:04:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f1b29585640 1 -- 172.21.15.44:0/3794673975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b240bf000 con 0x7f1b24072230 2024-04-22T07:04:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7ff13de7d640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1300e6c20 0x7ff1300e57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f6de359e640 1 -- start start 2024-04-22T07:04:21.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f6de359e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6de4157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f6de359e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de4150010 0x7f6de4153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f6de359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de41527d0 0x7f6de4153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f6de359e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6de406eae0 con 0x7f6de4150010 2024-04-22T07:04:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f6de359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6de415f050 con 0x7f6de41527d0 2024-04-22T07:04:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7f6de359e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6de4157470 con 0x7f6de40c7c30 2024-04-22T07:04:21.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fe7f57fa640 1 -- 172.21.15.44:0/1392244496 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8100c5840 con 0x7fe8100c9170 2024-04-22T07:04:21.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7feafdea2640 1 -- 172.21.15.44:0/3076660352 wait complete. 2024-04-22T07:04:21.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7feafdea2640 1 Processor -- start 2024-04-22T07:04:21.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.728+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3689841509 wait complete. 2024-04-22T07:04:21.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fb6e0a43640 1 Processor -- start 2024-04-22T07:04:21.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.731+0000 7fb6e0a43640 1 -- start start 2024-04-22T07:04:21.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fb6e0a43640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc06cf60 0x7fb6dc1492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.781 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fb6e0a43640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc0c4780 0x7fb6dc0cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fb6e0a43640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6dc151c70 0x7fb6dc08c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fb6e0a43640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc0c4c10 con 0x7fb6dc151c70 2024-04-22T07:04:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fb6e0a43640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc0c7530 con 0x7fb6dc06cf60 2024-04-22T07:04:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fb6e0a43640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc0c4d90 con 0x7fb6dc0c4780 2024-04-22T07:04:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fb6da59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc06cf60 0x7fb6dc1492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fb6da59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc06cf60 0x7fb6dc1492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36930/0 (socket says 172.21.15.44:36930) 2024-04-22T07:04:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fb6da59c640 1 -- 172.21.15.44:0/3603645257 learned_addr learned my addr 172.21.15.44:0/3603645257 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.732+0000 7fb6d9d9b640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc0c4780 0x7fb6dc0cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.782 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6da59c640 1 -- 172.21.15.44:0/3603645257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc0c4780 msgr2=0x7fb6dc0cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6da59c640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc0c4780 0x7fb6dc0cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6da59c640 1 -- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6dc151c70 msgr2=0x7fb6dc08c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6dad9d640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6dc151c70 0x7fb6dc08c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6da59c640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6dc151c70 0x7fb6dc08c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6da59c640 1 -- 172.21.15.44:0/3603645257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc0cc610 con 0x7fb6dc06cf60 2024-04-22T07:04:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6d9d9b640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc0c4780 0x7fb6dc0cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:21.783 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6dad9d640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6dc151c70 0x7fb6dc08c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7f384188c640 1 --2- 172.21.15.44:0/3547998499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3834045aa0 0x7f383404bea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f383c000b60 tx=0x7f383c0a7a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e518cc3ded47ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6da59c640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc06cf60 0x7fb6dc1492f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00d5f0 tx=0x7fb6cc00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7ff13d67c640 1 -- 172.21.15.44:0/670252588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1300dd4c0 msgr2=0x7ff13017ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.784 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7ff13d67c640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1300dd4c0 0x7ff13017ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7ff13d67c640 1 -- 172.21.15.44:0/670252588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1300e6c20 msgr2=0x7ff1300e57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7ff13d67c640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1300e6c20 0x7ff1300e57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7ff13d67c640 1 -- 172.21.15.44:0/670252588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1300dded0 con 0x7ff13004e6d0 2024-04-22T07:04:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7ff13d67c640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff13004e6d0 0x7ff1300ded80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff128009840 tx=0x7ff1280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.785 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7f6de1d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de4150010 0x7f6de4153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7f6de259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6de4157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7f6de259c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6de4157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:59160/0 (socket says 172.21.15.44:59160) 2024-04-22T07:04:21.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7f0677fff640 1 -- 172.21.15.44:0/2777032692 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0678018020 con 0x7f06840e3080 2024-04-22T07:04:21.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7f0677fff640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f067006c8f0 0x7f067006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7f6de259c640 1 -- 172.21.15.44:0/2831955646 learned_addr learned my addr 172.21.15.44:0/2831955646 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6caffd640 1 -- 172.21.15.44:0/3603645257 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc012b10 con 0x7fb6dc06cf60 2024-04-22T07:04:21.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6caffd640 1 -- 172.21.15.44:0/3603645257 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6cc012cb0 con 0x7fb6dc06cf60 2024-04-22T07:04:21.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6caffd640 1 -- 172.21.15.44:0/3603645257 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc012e60 con 0x7fb6dc06cf60 2024-04-22T07:04:21.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3603645257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6dc149f60 con 0x7fb6dc06cf60 2024-04-22T07:04:21.787 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3603645257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6dc149c80 con 0x7fb6dc06cf60 2024-04-22T07:04:21.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7f6de259c640 1 -- 172.21.15.44:0/2831955646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de41527d0 msgr2=0x7f6de4153440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T07:04:21.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7f6de259c640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de41527d0 0x7f6de4153440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7f6de259c640 1 -- 172.21.15.44:0/2831955646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de4150010 msgr2=0x7f6de4153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7f1b29585640 1 -- 172.21.15.44:0/3794673975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b240bf240 con 0x7f1b24072230 2024-04-22T07:04:21.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.734+0000 7feafdea2640 1 -- start start 2024-04-22T07:04:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7f6de259c640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de4150010 0x7f6de4153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7f6de259c640 1 -- 172.21.15.44:0/2831955646 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de41585f0 con 0x7f6de40c7c30 2024-04-22T07:04:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7feafdea2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c7080 0x7feaf815e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7feafdea2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf80c74a0 0x7feaf80cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7ff12dffb640 1 -- 172.21.15.44:0/670252588 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128042020 con 0x7ff13004e6d0 2024-04-22T07:04:21.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7feafdea2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf8159150 0x7feaf81fd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1300520d0 con 0x7ff13004e6d0 2024-04-22T07:04:21.790 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7f6de259c640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6de4157900 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40015a0 tx=0x7f6dd4001a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf80c74a0 0x7feaf80cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7f068b7fe640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f067006c8f0 0x7f067006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7f0677fff640 1 -- 172.21.15.44:0/2777032692 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f067808b9f0 con 0x7f06840e3080 2024-04-22T07:04:21.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7feafcea0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c7080 0x7feaf815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.791 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7f068b7fe640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f067006c8f0 0x7f067006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0680000f60 tx=0x7f0680009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf80c74a0 0x7feaf80cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:36946/0 (socket says 172.21.15.44:36946) 2024-04-22T07:04:21.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7feaf7fff640 1 -- 172.21.15.44:0/2627584784 learned_addr learned my addr 172.21.15.44:0/2627584784 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.735+0000 7fe7f77fe640 1 -- 172.21.15.44:0/1392244496 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8040035f0 con 0x7fe8100c9170 2024-04-22T07:04:21.792 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7f384108b640 1 -- 172.21.15.44:0/3547998499 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0a98e0 con 0x7f3834045aa0 2024-04-22T07:04:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7f384108b640 1 -- 172.21.15.44:0/3547998499 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383c0a9a80 con 0x7f3834045aa0 2024-04-22T07:04:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7f384108b640 1 -- 172.21.15.44:0/3547998499 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0b3400 con 0x7f3834045aa0 2024-04-22T07:04:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13017c210 con 0x7ff13004e6d0 2024-04-22T07:04:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7feaf806c5f0 con 0x7feaf8159150 2024-04-22T07:04:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8159530 con 0x7feaf80c74a0 2024-04-22T07:04:21.793 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7feaf815ee50 con 0x7feaf80c7080 2024-04-22T07:04:21.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7fe7f77fe640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe7fc06cd20 0x7fe7fc06f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7fe7f77fe640 1 -- 172.21.15.44:0/1392244496 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe8040050d0 con 0x7fe8100c9170 2024-04-22T07:04:21.794 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7feafcea0640 1 -- 172.21.15.44:0/2627584784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf80c74a0 msgr2=0x7feaf80cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7feafcea0640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf80c74a0 0x7feaf80cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7feafcea0640 1 -- 172.21.15.44:0/2627584784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf8159150 msgr2=0x7feaf81fd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7feafcea0640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf8159150 0x7feaf81fd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7fe80effd640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe7fc06cd20 0x7fe7fc06f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7feafcea0640 1 -- 172.21.15.44:0/2627584784 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf82009c0 con 0x7feaf80c7080 2024-04-22T07:04:21.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7feaf7fff640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf80c74a0 0x7feaf80cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:21.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7f1b0b7fe640 1 -- 172.21.15.44:0/3794673975 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b1c014440 con 0x7f1b24072230 2024-04-22T07:04:21.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7ff12dffb640 1 -- 172.21.15.44:0/670252588 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12803d470 con 0x7ff13004e6d0 2024-04-22T07:04:21.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.736+0000 7ff12dffb640 1 -- 172.21.15.44:0/670252588 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128037040 con 0x7ff13004e6d0 2024-04-22T07:04:21.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3603645257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6dc04fce0 con 0x7fb6dc06cf60 2024-04-22T07:04:21.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7f384308f640 1 -- 172.21.15.44:0/3547998499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3834045aa0 msgr2=0x7f383404bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7f384308f640 1 --2- 172.21.15.44:0/3547998499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3834045aa0 0x7f383404bea0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f383c000b60 tx=0x7f383c0a7a60 comp rx=0 tx=0).stop 2024-04-22T07:04:21.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7f1b0b7fe640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b1006c9f0 0x7f1b1006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7fe80effd640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe7fc06cd20 0x7fe7fc06f1e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe808035000 tx=0x7fe808030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff130005710 con 0x7ff13004e6d0 2024-04-22T07:04:21.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7f384308f640 1 -- 172.21.15.44:0/3547998499 shutdown_connections 2024-04-22T07:04:21.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7f384308f640 1 --2- 172.21.15.44:0/3547998499 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f3834045aa0 0x7f383404bea0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7f384308f640 1 --2- 172.21.15.44:0/3547998499 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f38340dd4d0 0x7f3834045560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7f384308f640 1 -- 172.21.15.44:0/3547998499 >> 172.21.15.44:0/3547998499 conn(0x7f38340505c0 msgr2=0x7f38340509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7f1b227fc640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b1006c9f0 0x7f1b1006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7f1b0b7fe640 1 -- 172.21.15.44:0/3794673975 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1b1c08ba10 con 0x7f1b24072230 2024-04-22T07:04:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.737+0000 7f384308f640 1 -- 172.21.15.44:0/3547998499 shutdown_connections 2024-04-22T07:04:21.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.738+0000 7feafcea0640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c7080 0x7feaf815e240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c5f0 tx=0x7feaec00cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.738+0000 7fb6caffd640 1 -- 172.21.15.44:0/3603645257 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb6cc005990 con 0x7fb6dc06cf60 2024-04-22T07:04:21.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.738+0000 7f384308f640 1 -- 172.21.15.44:0/3547998499 wait complete. 2024-04-22T07:04:21.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.738+0000 7f1b227fc640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b1006c9f0 0x7f1b1006eeb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b18036000 tx=0x7f1b18008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.738+0000 7ff12dffb640 1 -- 172.21.15.44:0/670252588 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff128045020 con 0x7ff13004e6d0 2024-04-22T07:04:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7fb6caffd640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb6bc06c8f0 0x7fb6bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7fb6caffd640 1 -- 172.21.15.44:0/3603645257 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb6cc00f020 con 0x7fb6dc06cf60 2024-04-22T07:04:21.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7fb6d9d9b640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb6bc06c8f0 0x7fb6bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7ff12dffb640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff11c06c9f0 0x7ff11c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7ff12dffb640 1 -- 172.21.15.44:0/670252588 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff1280adcc0 con 0x7ff13004e6d0 2024-04-22T07:04:21.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7f384308f640 1 Processor -- start 2024-04-22T07:04:21.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7f384308f640 1 -- start start 2024-04-22T07:04:21.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7ff13ce7b640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff11c06c9f0 0x7ff11c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7feaf57fa640 1 -- 172.21.15.44:0/2627584784 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec015070 con 0x7feaf80c7080 2024-04-22T07:04:21.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7fb6d9d9b640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb6bc06c8f0 0x7fb6bc06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005880 tx=0x7fb6d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf8200c60 con 0x7feaf80c7080 2024-04-22T07:04:21.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf8201220 con 0x7feaf80c7080 2024-04-22T07:04:21.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7feaf57fa640 1 -- 172.21.15.44:0/2627584784 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaec011800 con 0x7feaf80c7080 2024-04-22T07:04:21.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7f0677fff640 1 -- 172.21.15.44:0/2777032692 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f067801f020 con 0x7f06840e3080 2024-04-22T07:04:21.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.740+0000 7ff13ce7b640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff11c06c9f0 0x7ff11c06eeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1300de930 tx=0x7ff134009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.740+0000 7feaf57fa640 1 -- 172.21.15.44:0/2627584784 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec015070 con 0x7feaf80c7080 2024-04-22T07:04:21.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7f6dd2ffd640 1 -- 172.21.15.44:0/2831955646 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400e740 con 0x7f6de40c7c30 2024-04-22T07:04:21.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7f6dd2ffd640 1 -- 172.21.15.44:0/2831955646 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd4018070 con 0x7f6de40c7c30 2024-04-22T07:04:21.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7f6de359e640 1 -- 172.21.15.44:0/2831955646 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de41588b0 con 0x7f6de40c7c30 2024-04-22T07:04:21.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7f6dd2ffd640 1 -- 172.21.15.44:0/2831955646 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400eb00 con 0x7f6de40c7c30 2024-04-22T07:04:21.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.739+0000 7f6de359e640 1 -- 172.21.15.44:0/2831955646 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de4157100 con 0x7f6de40c7c30 2024-04-22T07:04:21.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f6dd0ff9640 1 -- 172.21.15.44:0/2831955646 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dbc005290 con 0x7f6de40c7c30 2024-04-22T07:04:21.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf82018e0 con 0x7feaf80c7080 2024-04-22T07:04:21.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384308f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3834045aa0 0x7f38340d9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384308f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f383404c3e0 0x7f383404a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384308f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38340dd4d0 0x7f383404eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384308f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f3834045ff0 con 0x7f383404c3e0 2024-04-22T07:04:21.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384308f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f38340e3d70 con 0x7f38340dd4d0 2024-04-22T07:04:21.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384308f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f38340e3120 con 0x7f3834045aa0 2024-04-22T07:04:21.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384188c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f383404c3e0 0x7f383404a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384188c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f383404c3e0 0x7f383404a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38112/0 (socket says 172.21.15.44:38112) 2024-04-22T07:04:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384188c640 1 -- 172.21.15.44:0/3592161907 learned_addr learned my addr 172.21.15.44:0/3592161907 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384208d640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3834045aa0 0x7f38340d9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384288e640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38340dd4d0 0x7f383404eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f6dd2ffd640 1 -- 172.21.15.44:0/2831955646 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dd400eca0 con 0x7f6de40c7c30 2024-04-22T07:04:21.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7feaf57fa640 1 -- 172.21.15.44:0/2627584784 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feaec018070 con 0x7feaf80c7080 2024-04-22T07:04:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7fe7f77fe640 1 -- 172.21.15.44:0/1392244496 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8100c5840 con 0x7fe8100c9170 2024-04-22T07:04:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384208d640 1 -- 172.21.15.44:0/3592161907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38340dd4d0 msgr2=0x7f383404eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384208d640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38340dd4d0 0x7f383404eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384208d640 1 -- 172.21.15.44:0/3592161907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f383404c3e0 msgr2=0x7f383404a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384208d640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f383404c3e0 0x7f383404a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:21.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384208d640 1 -- 172.21.15.44:0/3592161907 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383416bba0 con 0x7f3834045aa0 2024-04-22T07:04:21.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f6dd2ffd640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6dc406c970 0x7f6dc406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7feaf57fa640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feadc06caf0 0x7feadc06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7feaf7fff640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feadc06caf0 0x7feadc06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.810 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.743+0000 7feaf57fa640 1 -- 172.21.15.44:0/2627584784 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7feaec08b5b0 con 0x7feaf80c7080 2024-04-22T07:04:21.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f384208d640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3834045aa0 0x7f38340d9fd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f383800e8a0 tx=0x7f383800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f6dd2ffd640 1 -- 172.21.15.44:0/2831955646 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6dd408b810 con 0x7f6de40c7c30 2024-04-22T07:04:21.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.741+0000 7f6de1d9b640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6dc406c970 0x7f6dc406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.743+0000 7f6de1d9b640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6dc406c970 0x7f6dc406ee30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8036000 tx=0x7f6dd8009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.811 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.743+0000 7feaf7fff640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feadc06caf0 0x7feadc06efb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feaf81580e0 tx=0x7feae4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.744+0000 7f1b0b7fe640 1 -- 172.21.15.44:0/3794673975 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b240bf240 con 0x7f1b24072230 2024-04-22T07:04:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.746+0000 7feaf57fa640 1 -- 172.21.15.44:0/2627584784 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feaf82018e0 con 0x7feaf80c7080 2024-04-22T07:04:21.812 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.746+0000 7fb6caffd640 1 -- 172.21.15.44:0/3603645257 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb6dc149f60 con 0x7fb6dc06cf60 2024-04-22T07:04:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.750+0000 7ff12dffb640 1 -- 172.21.15.44:0/670252588 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff12804c070 con 0x7ff13004e6d0 2024-04-22T07:04:21.813 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.751+0000 7f6dd2ffd640 1 -- 172.21.15.44:0/2831955646 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6dd40112e0 con 0x7f6de40c7c30 2024-04-22T07:04:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.751+0000 7f3832ffd640 1 -- 172.21.15.44:0/3592161907 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838017070 con 0x7f3834045aa0 2024-04-22T07:04:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.751+0000 7f3832ffd640 1 -- 172.21.15.44:0/3592161907 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f383800c940 con 0x7f3834045aa0 2024-04-22T07:04:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.751+0000 7f384308f640 1 -- 172.21.15.44:0/3592161907 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38340da8b0 con 0x7f3834045aa0 2024-04-22T07:04:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.751+0000 7f384308f640 1 -- 172.21.15.44:0/3592161907 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f38341797e0 con 0x7f3834045aa0 2024-04-22T07:04:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.751+0000 7f3832ffd640 1 -- 172.21.15.44:0/3592161907 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838009e60 con 0x7f3834045aa0 2024-04-22T07:04:21.814 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.753+0000 7f3830ff9640 1 -- 172.21.15.44:0/3592161907 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3834005700 con 0x7f3834045aa0 2024-04-22T07:04:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.754+0000 7f3832ffd640 1 -- 172.21.15.44:0/3592161907 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f383801a070 con 0x7f3834045aa0 2024-04-22T07:04:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.755+0000 7f3832ffd640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f382006c9c0 0x7f382006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.755+0000 7f3832ffd640 1 -- 172.21.15.44:0/3592161907 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f383808bc40 con 0x7f3834045aa0 2024-04-22T07:04:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.756+0000 7f384188c640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f382006c9c0 0x7f382006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.757+0000 7f384188c640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f382006c9c0 0x7f382006ee80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f383c051230 tx=0x7f383c06e030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:21.815 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:21.759+0000 7f3832ffd640 1 -- 172.21.15.44:0/3592161907 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3838059a00 con 0x7f3834045aa0 2024-04-22T07:04:22.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.180+0000 7f0675ffb640 1 -- 172.21.15.44:0/2777032692 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0664005580 con 0x7f06840e3080 2024-04-22T07:04:22.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.181+0000 7f0677fff640 1 -- 172.21.15.44:0/2777032692 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f06780596a0 con 0x7f06840e3080 2024-04-22T07:04:22.179 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-04-22T07:04:22.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.182+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3603645257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb6dc06cd40 con 0x7fb6dc06cf60 2024-04-22T07:04:22.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.183+0000 7fb6caffd640 1 -- 172.21.15.44:0/3603645257 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb6cc05a240 con 0x7fb6dc06cf60 2024-04-22T07:04:22.181 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-04-22T07:04:22.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.184+0000 7f1b29585640 1 -- 172.21.15.44:0/3794673975 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1b2404fce0 con 0x7f1b24072230 2024-04-22T07:04:22.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.184+0000 7fe7f57fa640 1 -- 172.21.15.44:0/1392244496 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe81004fce0 con 0x7fe8100c9170 2024-04-22T07:04:22.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.184+0000 7f1b0b7fe640 1 -- 172.21.15.44:0/3794673975 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b1c059740 con 0x7f1b24072230 2024-04-22T07:04:22.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7fe7f77fe640 1 -- 172.21.15.44:0/1392244496 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe804058480 con 0x7fe8100c9170 2024-04-22T07:04:22.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 -- 172.21.15.44:0/2777032692 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f067006c8f0 msgr2=0x7f067006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f067006c8f0 0x7f067006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0680000f60 tx=0x7f0680009040 comp rx=0 tx=0).stop 2024-04-22T07:04:22.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 -- 172.21.15.44:0/2777032692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06840e3080 msgr2=0x7f0684179630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06840e3080 0x7f0684179630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f067800b410 tx=0x7f067800b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.184 INFO:teuthology.orchestra.run.smithi044.stdout:55834574856 2024-04-22T07:04:22.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 -- 172.21.15.44:0/2777032692 shutdown_connections 2024-04-22T07:04:22.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f067006c8f0 0x7f067006edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f06840e3080 0x7f0684179630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f06840dd3b0 0x7f06840dbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 --2- 172.21.15.44:0/2777032692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f068404ba50 0x7f06840cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 -- 172.21.15.44:0/2777032692 >> 172.21.15.44:0/2777032692 conn(0x7f0684050170 msgr2=0x7f06840d8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.185 INFO:teuthology.orchestra.run.smithi044.stdout:64424509447 2024-04-22T07:04:22.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 -- 172.21.15.44:0/2777032692 shutdown_connections 2024-04-22T07:04:22.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.185+0000 7f0675ffb640 1 -- 172.21.15.44:0/2777032692 wait complete. 2024-04-22T07:04:22.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.186+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb6bc06c8f0 msgr2=0x7fb6bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.186+0000 7fb6e0a43640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb6bc06c8f0 0x7fb6bc06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005880 tx=0x7fb6d0009040 comp rx=0 tx=0).stop 2024-04-22T07:04:22.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.186+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3603645257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc06cf60 msgr2=0x7fb6dc1492f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.186+0000 7fb6e0a43640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc06cf60 0x7fb6dc1492f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc00d5f0 tx=0x7fb6cc00dab0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.187+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3603645257 shutdown_connections 2024-04-22T07:04:22.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.187+0000 7fb6e0a43640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb6bc06c8f0 0x7fb6bc06edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.187+0000 7fb6e0a43640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb6dc0c4780 0x7fb6dc0cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.187+0000 7fb6e0a43640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb6dc151c70 0x7fb6dc08c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.187+0000 7fb6e0a43640 1 --2- 172.21.15.44:0/3603645257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6dc06cf60 0x7fb6dc1492f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.187+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3603645257 >> 172.21.15.44:0/3603645257 conn(0x7fb6dc072690 msgr2=0x7fb6dc0c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3603645257 shutdown_connections 2024-04-22T07:04:22.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7fb6e0a43640 1 -- 172.21.15.44:0/3603645257 wait complete. 2024-04-22T07:04:22.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7f1b29585640 1 -- 172.21.15.44:0/3794673975 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b1006c9f0 msgr2=0x7f1b1006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7f1b29585640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b1006c9f0 0x7f1b1006eeb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b18036000 tx=0x7f1b18008030 comp rx=0 tx=0).stop 2024-04-22T07:04:22.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7fe815281640 1 -- 172.21.15.44:0/1392244496 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe7fc06cd20 msgr2=0x7fe7fc06f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7fe815281640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe7fc06cd20 0x7fe7fc06f1e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe808035000 tx=0x7fe808030000 comp rx=0 tx=0).stop 2024-04-22T07:04:22.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7fe815281640 1 -- 172.21.15.44:0/1392244496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8100c9170 msgr2=0x7fe81015e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7fe815281640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8100c9170 0x7fe81015e310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe80400c3c0 tx=0x7fe80400c880 comp rx=0 tx=0).stop 2024-04-22T07:04:22.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7f1b29585640 1 -- 172.21.15.44:0/3794673975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24072230 msgr2=0x7f1b240be040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7f1b29585640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24072230 0x7f1b240be040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00bab0 tx=0x7f1b1c00bf70 comp rx=0 tx=0).stop 2024-04-22T07:04:22.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7f1b29585640 1 -- 172.21.15.44:0/3794673975 shutdown_connections 2024-04-22T07:04:22.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7f1b29585640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1b1006c9f0 0x7f1b1006eeb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7f1b29585640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1b24150ed0 0x7f1b240c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.188+0000 7f1b29585640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b240be630 0x7f1b241f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7f1b29585640 1 --2- 172.21.15.44:0/3794673975 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1b24072230 0x7f1b240be040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7f1b29585640 1 -- 172.21.15.44:0/3794673975 >> 172.21.15.44:0/3794673975 conn(0x7f1b240cc020 msgr2=0x7f1b2414dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7fe815281640 1 -- 172.21.15.44:0/1392244496 shutdown_connections 2024-04-22T07:04:22.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7fe815281640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe7fc06cd20 0x7fe7fc06f1e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7fe815281640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe81013eea0 0x7fe81015e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7fe815281640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8100c9170 0x7fe81015e310 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7fe815281640 1 --2- 172.21.15.44:0/1392244496 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe81006cd00 0x7fe8100cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7fe815281640 1 -- 172.21.15.44:0/1392244496 >> 172.21.15.44:0/1392244496 conn(0x7fe81006b970 msgr2=0x7fe8100c71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7fe815281640 1 -- 172.21.15.44:0/1392244496 shutdown_connections 2024-04-22T07:04:22.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7fe815281640 1 -- 172.21.15.44:0/1392244496 wait complete. 2024-04-22T07:04:22.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7f1b29585640 1 -- 172.21.15.44:0/3794673975 shutdown_connections 2024-04-22T07:04:22.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.189+0000 7f1b29585640 1 -- 172.21.15.44:0/3794673975 wait complete. 2024-04-22T07:04:22.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.190+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff13004d460 con 0x7ff13004e6d0 2024-04-22T07:04:22.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.191+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7feaf81fd510 con 0x7feaf80c7080 2024-04-22T07:04:22.191 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.191+0000 7ff12dffb640 1 -- 172.21.15.44:0/670252588 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff12807c0d0 con 0x7ff13004e6d0 2024-04-22T07:04:22.191 INFO:teuthology.orchestra.run.smithi044.stdout:64424509448 2024-04-22T07:04:22.192 INFO:teuthology.orchestra.run.smithi044.stdout:64424509449 2024-04-22T07:04:22.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.191+0000 7feaf57fa640 1 -- 172.21.15.44:0/2627584784 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feaf81fd510 con 0x7feaf80c7080 2024-04-22T07:04:22.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.195+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feadc06caf0 msgr2=0x7feadc06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.195+0000 7feafdea2640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feadc06caf0 0x7feadc06efb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feaf81580e0 tx=0x7feae4008040 comp rx=0 tx=0).stop 2024-04-22T07:04:22.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.195+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c7080 msgr2=0x7feaf815e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.193 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.195+0000 7feafdea2640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c7080 0x7feaf815e240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c5f0 tx=0x7feaec00cab0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.195+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff11c06c9f0 msgr2=0x7ff11c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.195+0000 7ff13e67e640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff11c06c9f0 0x7ff11c06eeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1300de930 tx=0x7ff134009040 comp rx=0 tx=0).stop 2024-04-22T07:04:22.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.195+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff13004e6d0 msgr2=0x7ff1300ded80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.195+0000 7ff13e67e640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff13004e6d0 0x7ff1300ded80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff128009840 tx=0x7ff1280046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.195+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 shutdown_connections 2024-04-22T07:04:22.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.195+0000 7ff13e67e640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff11c06c9f0 0x7ff11c06eeb0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff1300de930 tx=0x7ff134009040 comp rx=0 tx=0).stop 2024-04-22T07:04:22.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7ff13e67e640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff1300dd4c0 0x7ff13017ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.194 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7ff13e67e640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1300e6c20 0x7ff1300e57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7ff13e67e640 1 --2- 172.21.15.44:0/670252588 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff13004e6d0 0x7ff1300ded80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 >> 172.21.15.44:0/670252588 conn(0x7ff13000e6a0 msgr2=0x7ff1300e1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 shutdown_connections 2024-04-22T07:04:22.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7feafdea2640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feadc06caf0 0x7feadc06efb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7feafdea2640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7feaf8159150 0x7feaf81fd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7feafdea2640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf80c74a0 0x7feaf80cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7feafdea2640 1 --2- 172.21.15.44:0/2627584784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7feaf80c7080 0x7feaf815e240 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 >> 172.21.15.44:0/2627584784 conn(0x7feaf80c9db0 msgr2=0x7feaf8158350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 shutdown_connections 2024-04-22T07:04:22.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 shutdown_connections 2024-04-22T07:04:22.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7feafdea2640 1 -- 172.21.15.44:0/2627584784 wait complete. 2024-04-22T07:04:22.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7ff13e67e640 1 -- 172.21.15.44:0/670252588 wait complete. 2024-04-22T07:04:22.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.196+0000 7f3830ff9640 1 -- 172.21.15.44:0/3592161907 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3834033e90 con 0x7f3834045aa0 2024-04-22T07:04:22.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.198+0000 7f3832ffd640 1 -- 172.21.15.44:0/3592161907 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f383805dd00 con 0x7f3834045aa0 2024-04-22T07:04:22.198 INFO:teuthology.orchestra.run.smithi044.stdout:64424509449 2024-04-22T07:04:22.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.200+0000 7f6dd0ff9640 1 -- 172.21.15.44:0/2831955646 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6dbc005580 con 0x7f6de40c7c30 2024-04-22T07:04:22.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f6dd2ffd640 1 -- 172.21.15.44:0/2831955646 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6dd40112e0 con 0x7f6de40c7c30 2024-04-22T07:04:22.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f3830ff9640 1 -- 172.21.15.44:0/3592161907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f382006c9c0 msgr2=0x7f382006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f3830ff9640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f382006c9c0 0x7f382006ee80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f383c051230 tx=0x7f383c06e030 comp rx=0 tx=0).stop 2024-04-22T07:04:22.200 INFO:teuthology.orchestra.run.smithi044.stdout:64424509449 2024-04-22T07:04:22.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f3830ff9640 1 -- 172.21.15.44:0/3592161907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3834045aa0 msgr2=0x7f38340d9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f3830ff9640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3834045aa0 0x7f38340d9fd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f383800e8a0 tx=0x7f383800ed60 comp rx=0 tx=0).stop 2024-04-22T07:04:22.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f3830ff9640 1 -- 172.21.15.44:0/3592161907 shutdown_connections 2024-04-22T07:04:22.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f3830ff9640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f382006c9c0 0x7f382006ee80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f3830ff9640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38340dd4d0 0x7f383404eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f3830ff9640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f383404c3e0 0x7f383404a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f3830ff9640 1 --2- 172.21.15.44:0/3592161907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f3834045aa0 0x7f38340d9fd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.202+0000 7f3830ff9640 1 -- 172.21.15.44:0/3592161907 >> 172.21.15.44:0/3592161907 conn(0x7f38340505c0 msgr2=0x7f383404d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.203+0000 7f3830ff9640 1 -- 172.21.15.44:0/3592161907 shutdown_connections 2024-04-22T07:04:22.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.203+0000 7f3830ff9640 1 -- 172.21.15.44:0/3592161907 wait complete. 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 -- 172.21.15.44:0/2831955646 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6dc406c970 msgr2=0x7f6dc406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6dc406c970 0x7f6dc406ee30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8036000 tx=0x7f6dd8009030 comp rx=0 tx=0).stop 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 -- 172.21.15.44:0/2831955646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 msgr2=0x7f6de4157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6de4157900 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40015a0 tx=0x7f6dd4001a60 comp rx=0 tx=0).stop 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 -- 172.21.15.44:0/2831955646 shutdown_connections 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6dc406c970 0x7f6dc406ee30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de41527d0 0x7f6de4153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6de4150010 0x7f6de4153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 --2- 172.21.15.44:0/2831955646 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6de40c7c30 0x7f6de4157900 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 -- 172.21.15.44:0/2831955646 >> 172.21.15.44:0/2831955646 conn(0x7f6de40cc8b0 msgr2=0x7f6de40c90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 -- 172.21.15.44:0/2831955646 shutdown_connections 2024-04-22T07:04:22.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.206+0000 7f6dd0ff9640 1 -- 172.21.15.44:0/2831955646 wait complete. 2024-04-22T07:04:22.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.1 2024-04-22T07:04:22.206 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:22.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2024-04-22T07:04:22.211 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:22.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-04-22T07:04:22.212 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:22.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-04-22T07:04:22.212 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:22.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.4 2024-04-22T07:04:22.215 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:22.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.5 2024-04-22T07:04:22.217 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:22.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2024-04-22T07:04:22.218 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:22.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.7 2024-04-22T07:04:22.221 DEBUG:teuthology.parallel:result is None 2024-04-22T07:04:22.222 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-22T07:04:22.222 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:22.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.364+0000 7fe8817b6640 1 Processor -- start 2024-04-22T07:04:22.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.364+0000 7fe8817b6640 1 -- start start 2024-04-22T07:04:22.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe8817b6640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe87c154f50 0x7fe87c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe8817b6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe87c072c20 0x7fe87c0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe8817b6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe87c0cb1f0 con 0x7fe87c0d0d10 2024-04-22T07:04:22.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe8817b6640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fe87c0c5420 con 0x7fe87c154f50 2024-04-22T07:04:22.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe8817b6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe87c0c6540 con 0x7fe87c072c20 2024-04-22T07:04:22.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe87affd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe87c154f50 0x7fe87c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe87affd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe87c154f50 0x7fe87c15ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38118/0 (socket says 172.21.15.44:38118) 2024-04-22T07:04:22.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe87affd640 1 -- 172.21.15.44:0/3343903996 learned_addr learned my addr 172.21.15.44:0/3343903996 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:22.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe87a7fc640 1 --2- 172.21.15.44:0/3343903996 >> v2:172.21.15.70:3300/0 conn(0x7fe87c072c20 0x7fe87c0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe87affd640 1 -- 172.21.15.44:0/3343903996 >> v2:172.21.15.70:3300/0 conn(0x7fe87c072c20 msgr2=0x7fe87c0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:22.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe87affd640 1 --2- 172.21.15.44:0/3343903996 >> v2:172.21.15.70:3300/0 conn(0x7fe87c072c20 0x7fe87c0d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe87affd640 1 -- 172.21.15.44:0/3343903996 >> v1:172.21.15.70:6789/0 conn(0x7fe87c0d0d10 legacy=0x7fe87c15c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe87affd640 1 -- 172.21.15.44:0/3343903996 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87c155330 con 0x7fe87c154f50 2024-04-22T07:04:22.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.365+0000 7fe87a7fc640 1 --2- 172.21.15.44:0/3343903996 >> v2:172.21.15.70:3300/0 conn(0x7fe87c072c20 0x7fe87c0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:22.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.366+0000 7fe87affd640 1 --2- 172.21.15.44:0/3343903996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c154f50 0x7fe87c15ec90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0cdf60 tx=0x7fe87402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed4602a569a4f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:22.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.366+0000 7fe879ffb640 1 -- 172.21.15.44:0/3343903996 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87402f850 con 0x7fe87c154f50 2024-04-22T07:04:22.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.366+0000 7fe879ffb640 1 -- 172.21.15.44:0/3343903996 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87402f9f0 con 0x7fe87c154f50 2024-04-22T07:04:22.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.366+0000 7fe879ffb640 1 -- 172.21.15.44:0/3343903996 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87402fdc0 con 0x7fe87c154f50 2024-04-22T07:04:22.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.367+0000 7fe8817b6640 1 -- 172.21.15.44:0/3343903996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c154f50 msgr2=0x7fe87c15ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.367+0000 7fe8817b6640 1 --2- 172.21.15.44:0/3343903996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c154f50 0x7fe87c15ec90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0cdf60 tx=0x7fe87402ba30 comp rx=0 tx=0).stop 2024-04-22T07:04:22.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.367+0000 7fe8817b6640 1 -- 172.21.15.44:0/3343903996 shutdown_connections 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.367+0000 7fe8817b6640 1 --2- 172.21.15.44:0/3343903996 >> v2:172.21.15.70:3300/0 conn(0x7fe87c072c20 0x7fe87c0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.367+0000 7fe8817b6640 1 --2- 172.21.15.44:0/3343903996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c154f50 0x7fe87c15ec90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.367+0000 7fe8817b6640 1 -- 172.21.15.44:0/3343903996 >> 172.21.15.44:0/3343903996 conn(0x7fe87c06caa0 msgr2=0x7fe87c06ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.367+0000 7fe8817b6640 1 -- 172.21.15.44:0/3343903996 shutdown_connections 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.367+0000 7fe8817b6640 1 -- 172.21.15.44:0/3343903996 wait complete. 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.368+0000 7fe8817b6640 1 Processor -- start 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.368+0000 7fe8817b6640 1 -- start start 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe8817b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c072c20 0x7fe87c1f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe8817b6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe87c0d0d10 0x7fe87c15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe8817b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe87c154f50 0x7fe87c1f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe8817b6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c0c5c10 con 0x7fe87c072c20 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe8817b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe87c0c5930 con 0x7fe87c154f50 2024-04-22T07:04:22.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe8817b6640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe87c0c5ef0 con 0x7fe87c0d0d10 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe87affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c072c20 0x7fe87c1f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe87affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c072c20 0x7fe87c1f1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38134/0 (socket says 172.21.15.44:38134) 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe87affd640 1 -- 172.21.15.44:0/1121269179 learned_addr learned my addr 172.21.15.44:0/1121269179 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe87a7fc640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe87c0d0d10 0x7fe87c15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe87b7fe640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe87c154f50 0x7fe87c1f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe87affd640 1 -- 172.21.15.44:0/1121269179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe87c0d0d10 msgr2=0x7fe87c15a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe87affd640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe87c0d0d10 0x7fe87c15a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe87affd640 1 -- 172.21.15.44:0/1121269179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe87c154f50 msgr2=0x7fe87c1f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe87affd640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe87c154f50 0x7fe87c1f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.369+0000 7fe87affd640 1 -- 172.21.15.44:0/1121269179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87c1f09d0 con 0x7fe87c072c20 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.370+0000 7fe87affd640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c072c20 0x7fe87c1f1470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe874000c00 tx=0x7fe874004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.370+0000 7fe8637fe640 1 -- 172.21.15.44:0/1121269179 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874047070 con 0x7fe87c072c20 2024-04-22T07:04:22.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.370+0000 7fe8637fe640 1 -- 172.21.15.44:0/1121269179 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe874037cb0 con 0x7fe87c072c20 2024-04-22T07:04:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.370+0000 7fe8817b6640 1 -- 172.21.15.44:0/1121269179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87c1f46e0 con 0x7fe87c072c20 2024-04-22T07:04:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.370+0000 7fe8637fe640 1 -- 172.21.15.44:0/1121269179 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874037e60 con 0x7fe87c072c20 2024-04-22T07:04:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.370+0000 7fe8817b6640 1 -- 172.21.15.44:0/1121269179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe87c1f0c70 con 0x7fe87c072c20 2024-04-22T07:04:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.371+0000 7fe8817b6640 1 -- 172.21.15.44:0/1121269179 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe87c150a00 con 0x7fe87c072c20 2024-04-22T07:04:22.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.371+0000 7fe8637fe640 1 -- 172.21.15.44:0/1121269179 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe87404a070 con 0x7fe87c072c20 2024-04-22T07:04:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.372+0000 7fe8637fe640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe86806c8f0 0x7fe86806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.372+0000 7fe87a7fc640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe86806c8f0 0x7fe86806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.372+0000 7fe8637fe640 1 -- 172.21.15.44:0/1121269179 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe8740adee0 con 0x7fe87c072c20 2024-04-22T07:04:22.372 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.372+0000 7fe87a7fc640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe86806c8f0 0x7fe86806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0ca770 tx=0x7fe870008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:22.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.376+0000 7fe8637fe640 1 -- 172.21.15.44:0/1121269179 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe874033030 con 0x7fe87c072c20 2024-04-22T07:04:22.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.557+0000 7fe8817b6640 1 -- 172.21.15.44:0/1121269179 --> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe87c0c61c0 con 0x7fe86806c8f0 2024-04-22T07:04:22.730 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.563+0000 7fe8637fe640 1 -- 172.21.15.44:0/1121269179 <== mgr.4101 v2:172.21.15.44:6832/1889438723 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251859 (secure 0 0 0) 0x7fe87c0c61c0 con 0x7fe86806c8f0 2024-04-22T07:04:22.737 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:22.738 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T07:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 -- 172.21.15.44:0/1121269179 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe86806c8f0 msgr2=0x7fe86806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe86806c8f0 0x7fe86806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0ca770 tx=0x7fe870008040 comp rx=0 tx=0).stop 2024-04-22T07:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 -- 172.21.15.44:0/1121269179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c072c20 msgr2=0x7fe87c1f1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c072c20 0x7fe87c1f1470 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe874000c00 tx=0x7fe874004730 comp rx=0 tx=0).stop 2024-04-22T07:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 -- 172.21.15.44:0/1121269179 shutdown_connections 2024-04-22T07:04:22.741 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe86806c8f0 0x7fe86806edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe87c0d0d10 0x7fe87c15a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe87c154f50 0x7fe87c1f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 --2- 172.21.15.44:0/1121269179 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe87c072c20 0x7fe87c1f1470 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 -- 172.21.15.44:0/1121269179 >> 172.21.15.44:0/1121269179 conn(0x7fe87c06caa0 msgr2=0x7fe87c14ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 -- 172.21.15.44:0/1121269179 shutdown_connections 2024-04-22T07:04:22.742 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.566+0000 7fe8817b6640 1 -- 172.21.15.44:0/1121269179 wait complete. 2024-04-22T07:04:22.742 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-22T07:04:21.713176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239976,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734744,"statfs":{"total":773094113280,"available":772848377856,"internally_reserved":0,"allocated":3121152,"data_stored":1835436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003167"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961498+0000","last_change":"2024-04-22T07:04:13.800864+0000","last_active":"2024-04-22T07:04:14.961498+0000","last_peered":"2024-04-22T07:04:14.961498+0000","last_clean":"2024-04-22T07:04:14.961498+0000","last_became_active":"2024-04-22T07:04:13.800611+0000","last_became_peered":"2024-04-22T07:04:13.800611+0000","last_unstale":"2024-04-22T07:04:14.961498+0000","last_undegraded":"2024-04-22T07:04:14.961498+0000","last_fullsized":"2024-04-22T07:04:14.961498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:55.129127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056077+0000","last_change":"2024-04-22T07:04:13.798390+0000","last_active":"2024-04-22T07:04:15.056077+0000","last_peered":"2024-04-22T07:04:15.056077+0000","last_clean":"2024-04-22T07:04:15.056077+0000","last_became_active":"2024-04-22T07:04:13.797927+0000","last_became_peered":"2024-04-22T07:04:13.797927+0000","last_unstale":"2024-04-22T07:04:15.056077+0000","last_undegraded":"2024-04-22T07:04:15.056077+0000","last_fullsized":"2024-04-22T07:04:15.056077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056425+0000","last_change":"2024-04-22T07:04:13.795288+0000","last_active":"2024-04-22T07:04:15.056425+0000","last_peered":"2024-04-22T07:04:15.056425+0000","last_clean":"2024-04-22T07:04:15.056425+0000","last_became_active":"2024-04-22T07:04:13.794704+0000","last_became_peered":"2024-04-22T07:04:13.794704+0000","last_unstale":"2024-04-22T07:04:15.056425+0000","last_undegraded":"2024-04-22T07:04:15.056425+0000","last_fullsized":"2024-04-22T07:04:15.056425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783223+0000","last_change":"2024-04-22T07:04:13.801884+0000","last_active":"2024-04-22T07:04:14.783223+0000","last_peered":"2024-04-22T07:04:14.783223+0000","last_clean":"2024-04-22T07:04:14.783223+0000","last_became_active":"2024-04-22T07:04:13.801620+0000","last_became_peered":"2024-04-22T07:04:13.801620+0000","last_unstale":"2024-04-22T07:04:14.783223+0000","last_undegraded":"2024-04-22T07:04:14.783223+0000","last_fullsized":"2024-04-22T07:04:14.783223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961151+0000","last_change":"2024-04-22T07:04:13.798670+0000","last_active":"2024-04-22T07:04:14.961151+0000","last_peered":"2024-04-22T07:04:14.961151+0000","last_clean":"2024-04-22T07:04:14.961151+0000","last_became_active":"2024-04-22T07:04:13.798384+0000","last_became_peered":"2024-04-22T07:04:13.798384+0000","last_unstale":"2024-04-22T07:04:14.961151+0000","last_undegraded":"2024-04-22T07:04:14.961151+0000","last_fullsized":"2024-04-22T07:04:14.961151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783124+0000","last_change":"2024-04-22T07:04:13.796468+0000","last_active":"2024-04-22T07:04:14.783124+0000","last_peered":"2024-04-22T07:04:14.783124+0000","last_clean":"2024-04-22T07:04:14.783124+0000","last_became_active":"2024-04-22T07:04:13.796139+0000","last_became_peered":"2024-04-22T07:04:13.796139+0000","last_unstale":"2024-04-22T07:04:14.783124+0000","last_undegraded":"2024-04-22T07:04:14.783124+0000","last_fullsized":"2024-04-22T07:04:14.783124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961066+0000","last_change":"2024-04-22T07:04:13.801449+0000","last_active":"2024-04-22T07:04:14.961066+0000","last_peered":"2024-04-22T07:04:14.961066+0000","last_clean":"2024-04-22T07:04:14.961066+0000","last_became_active":"2024-04-22T07:04:13.799374+0000","last_became_peered":"2024-04-22T07:04:13.799374+0000","last_unstale":"2024-04-22T07:04:14.961066+0000","last_undegraded":"2024-04-22T07:04:14.961066+0000","last_fullsized":"2024-04-22T07:04:14.961066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054559+0000","last_change":"2024-04-22T07:04:13.802780+0000","last_active":"2024-04-22T07:04:15.054559+0000","last_peered":"2024-04-22T07:04:15.054559+0000","last_clean":"2024-04-22T07:04:15.054559+0000","last_became_active":"2024-04-22T07:04:13.802371+0000","last_became_peered":"2024-04-22T07:04:13.802371+0000","last_unstale":"2024-04-22T07:04:15.054559+0000","last_undegraded":"2024-04-22T07:04:15.054559+0000","last_fullsized":"2024-04-22T07:04:15.054559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961532+0000","last_change":"2024-04-22T07:04:13.792483+0000","last_active":"2024-04-22T07:04:14.961532+0000","last_peered":"2024-04-22T07:04:14.961532+0000","last_clean":"2024-04-22T07:04:14.961532+0000","last_became_active":"2024-04-22T07:04:13.792198+0000","last_became_peered":"2024-04-22T07:04:13.792198+0000","last_unstale":"2024-04-22T07:04:14.961532+0000","last_undegraded":"2024-04-22T07:04:14.961532+0000","last_fullsized":"2024-04-22T07:04:14.961532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:36.943428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782040+0000","last_change":"2024-04-22T07:04:13.797379+0000","last_active":"2024-04-22T07:04:14.782040+0000","last_peered":"2024-04-22T07:04:14.782040+0000","last_clean":"2024-04-22T07:04:14.782040+0000","last_became_active":"2024-04-22T07:04:13.797029+0000","last_became_peered":"2024-04-22T07:04:13.797029+0000","last_unstale":"2024-04-22T07:04:14.782040+0000","last_undegraded":"2024-04-22T07:04:14.782040+0000","last_fullsized":"2024-04-22T07:04:14.782040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781680+0000","last_change":"2024-04-22T07:04:13.789875+0000","last_active":"2024-04-22T07:04:14.781680+0000","last_peered":"2024-04-22T07:04:14.781680+0000","last_clean":"2024-04-22T07:04:14.781680+0000","last_became_active":"2024-04-22T07:04:13.789598+0000","last_became_peered":"2024-04-22T07:04:13.789598+0000","last_unstale":"2024-04-22T07:04:14.781680+0000","last_undegraded":"2024-04-22T07:04:14.781680+0000","last_fullsized":"2024-04-22T07:04:14.781680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055156+0000","last_change":"2024-04-22T07:04:13.791103+0000","last_active":"2024-04-22T07:04:15.055156+0000","last_peered":"2024-04-22T07:04:15.055156+0000","last_clean":"2024-04-22T07:04:15.055156+0000","last_became_active":"2024-04-22T07:04:13.790819+0000","last_became_peered":"2024-04-22T07:04:13.790819+0000","last_unstale":"2024-04-22T07:04:15.055156+0000","last_undegraded":"2024-04-22T07:04:15.055156+0000","last_fullsized":"2024-04-22T07:04:15.055156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:50.632020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055402+0000","last_change":"2024-04-22T07:04:13.802407+0000","last_active":"2024-04-22T07:04:15.055402+0000","last_peered":"2024-04-22T07:04:15.055402+0000","last_clean":"2024-04-22T07:04:15.055402+0000","last_became_active":"2024-04-22T07:04:13.801785+0000","last_became_peered":"2024-04-22T07:04:13.801785+0000","last_unstale":"2024-04-22T07:04:15.055402+0000","last_undegraded":"2024-04-22T07:04:15.055402+0000","last_fullsized":"2024-04-22T07:04:15.055402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055557+0000","last_change":"2024-04-22T07:04:13.795888+0000","last_active":"2024-04-22T07:04:15.055557+0000","last_peered":"2024-04-22T07:04:15.055557+0000","last_clean":"2024-04-22T07:04:15.055557+0000","last_became_active":"2024-04-22T07:04:13.795092+0000","last_became_peered":"2024-04-22T07:04:13.795092+0000","last_unstale":"2024-04-22T07:04:15.055557+0000","last_undegraded":"2024-04-22T07:04:15.055557+0000","last_fullsized":"2024-04-22T07:04:15.055557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:54.119208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055926+0000","last_change":"2024-04-22T07:04:13.797138+0000","last_active":"2024-04-22T07:04:15.055926+0000","last_peered":"2024-04-22T07:04:15.055926+0000","last_clean":"2024-04-22T07:04:15.055926+0000","last_became_active":"2024-04-22T07:04:13.796763+0000","last_became_peered":"2024-04-22T07:04:13.796763+0000","last_unstale":"2024-04-22T07:04:15.055926+0000","last_undegraded":"2024-04-22T07:04:15.055926+0000","last_fullsized":"2024-04-22T07:04:15.055926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782465+0000","last_change":"2024-04-22T07:04:13.803869+0000","last_active":"2024-04-22T07:04:14.782465+0000","last_peered":"2024-04-22T07:04:14.782465+0000","last_clean":"2024-04-22T07:04:14.782465+0000","last_became_active":"2024-04-22T07:04:13.803451+0000","last_became_peered":"2024-04-22T07:04:13.803451+0000","last_unstale":"2024-04-22T07:04:14.782465+0000","last_undegraded":"2024-04-22T07:04:14.782465+0000","last_fullsized":"2024-04-22T07:04:14.782465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783077+0000","last_change":"2024-04-22T07:04:13.795746+0000","last_active":"2024-04-22T07:04:14.783077+0000","last_peered":"2024-04-22T07:04:14.783077+0000","last_clean":"2024-04-22T07:04:14.783077+0000","last_became_active":"2024-04-22T07:04:13.794694+0000","last_became_peered":"2024-04-22T07:04:13.794694+0000","last_unstale":"2024-04-22T07:04:14.783077+0000","last_undegraded":"2024-04-22T07:04:14.783077+0000","last_fullsized":"2024-04-22T07:04:14.783077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054829+0000","last_change":"2024-04-22T07:04:13.790670+0000","last_active":"2024-04-22T07:04:15.054829+0000","last_peered":"2024-04-22T07:04:15.054829+0000","last_clean":"2024-04-22T07:04:15.054829+0000","last_became_active":"2024-04-22T07:04:13.790104+0000","last_became_peered":"2024-04-22T07:04:13.790104+0000","last_unstale":"2024-04-22T07:04:15.054829+0000","last_undegraded":"2024-04-22T07:04:15.054829+0000","last_fullsized":"2024-04-22T07:04:15.054829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:09.325971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961464+0000","last_change":"2024-04-22T07:04:13.799016+0000","last_active":"2024-04-22T07:04:14.961464+0000","last_peered":"2024-04-22T07:04:14.961464+0000","last_clean":"2024-04-22T07:04:14.961464+0000","last_became_active":"2024-04-22T07:04:13.798716+0000","last_became_peered":"2024-04-22T07:04:13.798716+0000","last_unstale":"2024-04-22T07:04:14.961464+0000","last_undegraded":"2024-04-22T07:04:14.961464+0000","last_fullsized":"2024-04-22T07:04:14.961464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:36.609382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781586+0000","last_change":"2024-04-22T07:04:13.797296+0000","last_active":"2024-04-22T07:04:14.781586+0000","last_peered":"2024-04-22T07:04:14.781586+0000","last_clean":"2024-04-22T07:04:14.781586+0000","last_became_active":"2024-04-22T07:04:13.796997+0000","last_became_peered":"2024-04-22T07:04:13.796997+0000","last_unstale":"2024-04-22T07:04:14.781586+0000","last_undegraded":"2024-04-22T07:04:14.781586+0000","last_fullsized":"2024-04-22T07:04:14.781586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055994+0000","last_change":"2024-04-22T07:04:13.799675+0000","last_active":"2024-04-22T07:04:15.055994+0000","last_peered":"2024-04-22T07:04:15.055994+0000","last_clean":"2024-04-22T07:04:15.055994+0000","last_became_active":"2024-04-22T07:04:13.799411+0000","last_became_peered":"2024-04-22T07:04:13.799411+0000","last_unstale":"2024-04-22T07:04:15.055994+0000","last_undegraded":"2024-04-22T07:04:15.055994+0000","last_fullsized":"2024-04-22T07:04:15.055994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783058+0000","last_change":"2024-04-22T07:04:13.796128+0000","last_active":"2024-04-22T07:04:14.783058+0000","last_peered":"2024-04-22T07:04:14.783058+0000","last_clean":"2024-04-22T07:04:14.783058+0000","last_became_active":"2024-04-22T07:04:13.795330+0000","last_became_peered":"2024-04-22T07:04:13.795330+0000","last_unstale":"2024-04-22T07:04:14.783058+0000","last_undegraded":"2024-04-22T07:04:14.783058+0000","last_fullsized":"2024-04-22T07:04:14.783058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055294+0000","last_change":"2024-04-22T07:04:13.800458+0000","last_active":"2024-04-22T07:04:15.055294+0000","last_peered":"2024-04-22T07:04:15.055294+0000","last_clean":"2024-04-22T07:04:15.055294+0000","last_became_active":"2024-04-22T07:04:13.800171+0000","last_became_peered":"2024-04-22T07:04:13.800171+0000","last_unstale":"2024-04-22T07:04:15.055294+0000","last_undegraded":"2024-04-22T07:04:15.055294+0000","last_fullsized":"2024-04-22T07:04:15.055294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782518+0000","last_change":"2024-04-22T07:04:13.802472+0000","last_active":"2024-04-22T07:04:14.782518+0000","last_peered":"2024-04-22T07:04:14.782518+0000","last_clean":"2024-04-22T07:04:14.782518+0000","last_became_active":"2024-04-22T07:04:13.802201+0000","last_became_peered":"2024-04-22T07:04:13.802201+0000","last_unstale":"2024-04-22T07:04:14.782518+0000","last_undegraded":"2024-04-22T07:04:14.782518+0000","last_fullsized":"2024-04-22T07:04:14.782518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055335+0000","last_change":"2024-04-22T07:04:13.795055+0000","last_active":"2024-04-22T07:04:15.055335+0000","last_peered":"2024-04-22T07:04:15.055335+0000","last_clean":"2024-04-22T07:04:15.055335+0000","last_became_active":"2024-04-22T07:04:13.794145+0000","last_became_peered":"2024-04-22T07:04:13.794145+0000","last_unstale":"2024-04-22T07:04:15.055335+0000","last_undegraded":"2024-04-22T07:04:15.055335+0000","last_fullsized":"2024-04-22T07:04:15.055335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782104+0000","last_change":"2024-04-22T07:04:13.799772+0000","last_active":"2024-04-22T07:04:14.782104+0000","last_peered":"2024-04-22T07:04:14.782104+0000","last_clean":"2024-04-22T07:04:14.782104+0000","last_became_active":"2024-04-22T07:04:13.799504+0000","last_became_peered":"2024-04-22T07:04:13.799504+0000","last_unstale":"2024-04-22T07:04:14.782104+0000","last_undegraded":"2024-04-22T07:04:14.782104+0000","last_fullsized":"2024-04-22T07:04:14.782104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056133+0000","last_change":"2024-04-22T07:04:13.804604+0000","last_active":"2024-04-22T07:04:15.056133+0000","last_peered":"2024-04-22T07:04:15.056133+0000","last_clean":"2024-04-22T07:04:15.056133+0000","last_became_active":"2024-04-22T07:04:13.803594+0000","last_became_peered":"2024-04-22T07:04:13.803594+0000","last_unstale":"2024-04-22T07:04:15.056133+0000","last_undegraded":"2024-04-22T07:04:15.056133+0000","last_fullsized":"2024-04-22T07:04:15.056133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055262+0000","last_change":"2024-04-22T07:04:13.795947+0000","last_active":"2024-04-22T07:04:15.055262+0000","last_peered":"2024-04-22T07:04:15.055262+0000","last_clean":"2024-04-22T07:04:15.055262+0000","last_became_active":"2024-04-22T07:04:13.795223+0000","last_became_peered":"2024-04-22T07:04:13.795223+0000","last_unstale":"2024-04-22T07:04:15.055262+0000","last_undegraded":"2024-04-22T07:04:15.055262+0000","last_fullsized":"2024-04-22T07:04:15.055262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056195+0000","last_change":"2024-04-22T07:04:13.804225+0000","last_active":"2024-04-22T07:04:15.056195+0000","last_peered":"2024-04-22T07:04:15.056195+0000","last_clean":"2024-04-22T07:04:15.056195+0000","last_became_active":"2024-04-22T07:04:13.803920+0000","last_became_peered":"2024-04-22T07:04:13.803920+0000","last_unstale":"2024-04-22T07:04:15.056195+0000","last_undegraded":"2024-04-22T07:04:15.056195+0000","last_fullsized":"2024-04-22T07:04:15.056195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783228+0000","last_change":"2024-04-22T07:04:13.796357+0000","last_active":"2024-04-22T07:04:14.783228+0000","last_peered":"2024-04-22T07:04:14.783228+0000","last_clean":"2024-04-22T07:04:14.783228+0000","last_became_active":"2024-04-22T07:04:13.795784+0000","last_became_peered":"2024-04-22T07:04:13.795784+0000","last_unstale":"2024-04-22T07:04:14.783228+0000","last_undegraded":"2024-04-22T07:04:14.783228+0000","last_fullsized":"2024-04-22T07:04:14.783228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054855+0000","last_change":"2024-04-22T07:04:13.784337+0000","last_active":"2024-04-22T07:04:15.054855+0000","last_peered":"2024-04-22T07:04:15.054855+0000","last_clean":"2024-04-22T07:04:15.054855+0000","last_became_active":"2024-04-22T07:04:13.783914+0000","last_became_peered":"2024-04-22T07:04:13.783914+0000","last_unstale":"2024-04-22T07:04:15.054855+0000","last_undegraded":"2024-04-22T07:04:15.054855+0000","last_fullsized":"2024-04-22T07:04:15.054855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:43:41.211395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781515+0000","last_change":"2024-04-22T07:04:13.806969+0000","last_active":"2024-04-22T07:04:14.781515+0000","last_peered":"2024-04-22T07:04:14.781515+0000","last_clean":"2024-04-22T07:04:14.781515+0000","last_became_active":"2024-04-22T07:04:13.806715+0000","last_became_peered":"2024-04-22T07:04:13.806715+0000","last_unstale":"2024-04-22T07:04:14.781515+0000","last_undegraded":"2024-04-22T07:04:14.781515+0000","last_fullsized":"2024-04-22T07:04:14.781515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054895+0000","last_change":"2024-04-22T07:04:13.794896+0000","last_active":"2024-04-22T07:04:15.054895+0000","last_peered":"2024-04-22T07:04:15.054895+0000","last_clean":"2024-04-22T07:04:15.054895+0000","last_became_active":"2024-04-22T07:04:13.794605+0000","last_became_peered":"2024-04-22T07:04:13.794605+0000","last_unstale":"2024-04-22T07:04:15.054895+0000","last_undegraded":"2024-04-22T07:04:15.054895+0000","last_fullsized":"2024-04-22T07:04:15.054895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:02.613130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.734312+0000","last_change":"2024-04-22T07:04:12.786738+0000","last_active":"2024-04-22T07:04:15.734312+0000","last_peered":"2024-04-22T07:04:15.734312+0000","last_clean":"2024-04-22T07:04:15.734312+0000","last_became_active":"2024-04-22T07:04:12.786413+0000","last_became_peered":"2024-04-22T07:04:12.786413+0000","last_unstale":"2024-04-22T07:04:15.734312+0000","last_undegraded":"2024-04-22T07:04:15.734312+0000","last_fullsized":"2024-04-22T07:04:15.734312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056196+0000","last_change":"2024-04-22T07:04:13.800525+0000","last_active":"2024-04-22T07:04:15.056196+0000","last_peered":"2024-04-22T07:04:15.056196+0000","last_clean":"2024-04-22T07:04:15.056196+0000","last_became_active":"2024-04-22T07:04:13.800254+0000","last_became_peered":"2024-04-22T07:04:13.800254+0000","last_unstale":"2024-04-22T07:04:15.056196+0000","last_undegraded":"2024-04-22T07:04:15.056196+0000","last_fullsized":"2024-04-22T07:04:15.056196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.733258+0000","last_change":"2024-04-22T07:04:12.786726+0000","last_active":"2024-04-22T07:04:15.733258+0000","last_peered":"2024-04-22T07:04:15.733258+0000","last_clean":"2024-04-22T07:04:15.733258+0000","last_became_active":"2024-04-22T07:04:12.786444+0000","last_became_peered":"2024-04-22T07:04:12.786444+0000","last_unstale":"2024-04-22T07:04:15.733258+0000","last_undegraded":"2024-04-22T07:04:15.733258+0000","last_fullsized":"2024-04-22T07:04:15.733258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055394+0000","last_change":"2024-04-22T07:04:12.787357+0000","last_active":"2024-04-22T07:04:15.055394+0000","last_peered":"2024-04-22T07:04:15.055394+0000","last_clean":"2024-04-22T07:04:15.055394+0000","last_became_active":"2024-04-22T07:04:12.787066+0000","last_became_peered":"2024-04-22T07:04:12.787066+0000","last_unstale":"2024-04-22T07:04:15.055394+0000","last_undegraded":"2024-04-22T07:04:15.055394+0000","last_fullsized":"2024-04-22T07:04:15.055394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054713+0000","last_change":"2024-04-22T07:04:13.807697+0000","last_active":"2024-04-22T07:04:15.054713+0000","last_peered":"2024-04-22T07:04:15.054713+0000","last_clean":"2024-04-22T07:04:15.054713+0000","last_became_active":"2024-04-22T07:04:13.807327+0000","last_became_peered":"2024-04-22T07:04:13.807327+0000","last_unstale":"2024-04-22T07:04:15.054713+0000","last_undegraded":"2024-04-22T07:04:15.054713+0000","last_fullsized":"2024-04-22T07:04:15.054713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783313+0000","last_change":"2024-04-22T07:04:12.787783+0000","last_active":"2024-04-22T07:04:14.783313+0000","last_peered":"2024-04-22T07:04:14.783313+0000","last_clean":"2024-04-22T07:04:14.783313+0000","last_became_active":"2024-04-22T07:04:12.786765+0000","last_became_peered":"2024-04-22T07:04:12.786765+0000","last_unstale":"2024-04-22T07:04:14.783313+0000","last_undegraded":"2024-04-22T07:04:14.783313+0000","last_fullsized":"2024-04-22T07:04:14.783313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055740+0000","last_change":"2024-04-22T07:04:13.804931+0000","last_active":"2024-04-22T07:04:15.055740+0000","last_peered":"2024-04-22T07:04:15.055740+0000","last_clean":"2024-04-22T07:04:15.055740+0000","last_became_active":"2024-04-22T07:04:13.804629+0000","last_became_peered":"2024-04-22T07:04:13.804629+0000","last_unstale":"2024-04-22T07:04:15.055740+0000","last_undegraded":"2024-04-22T07:04:15.055740+0000","last_fullsized":"2024-04-22T07:04:15.055740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781808+0000","last_change":"2024-04-22T07:04:12.789279+0000","last_active":"2024-04-22T07:04:14.781808+0000","last_peered":"2024-04-22T07:04:14.781808+0000","last_clean":"2024-04-22T07:04:14.781808+0000","last_became_active":"2024-04-22T07:04:12.787705+0000","last_became_peered":"2024-04-22T07:04:12.787705+0000","last_unstale":"2024-04-22T07:04:14.781808+0000","last_undegraded":"2024-04-22T07:04:14.781808+0000","last_fullsized":"2024-04-22T07:04:14.781808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782812+0000","last_change":"2024-04-22T07:04:13.794292+0000","last_active":"2024-04-22T07:04:14.782812+0000","last_peered":"2024-04-22T07:04:14.782812+0000","last_clean":"2024-04-22T07:04:14.782812+0000","last_became_active":"2024-04-22T07:04:13.793986+0000","last_became_peered":"2024-04-22T07:04:13.793986+0000","last_unstale":"2024-04-22T07:04:14.782812+0000","last_undegraded":"2024-04-22T07:04:14.782812+0000","last_fullsized":"2024-04-22T07:04:14.782812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782990+0000","last_change":"2024-04-22T07:04:12.787064+0000","last_active":"2024-04-22T07:04:14.782990+0000","last_peered":"2024-04-22T07:04:14.782990+0000","last_clean":"2024-04-22T07:04:14.782990+0000","last_became_active":"2024-04-22T07:04:12.786430+0000","last_became_peered":"2024-04-22T07:04:12.786430+0000","last_unstale":"2024-04-22T07:04:14.782990+0000","last_undegraded":"2024-04-22T07:04:14.782990+0000","last_fullsized":"2024-04-22T07:04:14.782990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056732+0000","last_change":"2024-04-22T07:04:13.801337+0000","last_active":"2024-04-22T07:04:15.056732+0000","last_peered":"2024-04-22T07:04:15.056732+0000","last_clean":"2024-04-22T07:04:15.056732+0000","last_became_active":"2024-04-22T07:04:13.800916+0000","last_became_peered":"2024-04-22T07:04:13.800916+0000","last_unstale":"2024-04-22T07:04:15.056732+0000","last_undegraded":"2024-04-22T07:04:15.056732+0000","last_fullsized":"2024-04-22T07:04:15.056732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.803158+0000","last_change":"2024-04-22T07:04:12.787741+0000","last_active":"2024-04-22T07:04:18.803158+0000","last_peered":"2024-04-22T07:04:18.803158+0000","last_clean":"2024-04-22T07:04:18.803158+0000","last_became_active":"2024-04-22T07:04:12.787405+0000","last_became_peered":"2024-04-22T07:04:12.787405+0000","last_unstale":"2024-04-22T07:04:18.803158+0000","last_undegraded":"2024-04-22T07:04:18.803158+0000","last_fullsized":"2024-04-22T07:04:18.803158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055184+0000","last_change":"2024-04-22T07:04:13.795502+0000","last_active":"2024-04-22T07:04:15.055184+0000","last_peered":"2024-04-22T07:04:15.055184+0000","last_clean":"2024-04-22T07:04:15.055184+0000","last_became_active":"2024-04-22T07:04:13.795199+0000","last_became_peered":"2024-04-22T07:04:13.795199+0000","last_unstale":"2024-04-22T07:04:15.055184+0000","last_undegraded":"2024-04-22T07:04:15.055184+0000","last_fullsized":"2024-04-22T07:04:15.055184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:07.780008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.802058+0000","last_change":"2024-04-22T07:04:12.788572+0000","last_active":"2024-04-22T07:04:18.802058+0000","last_peered":"2024-04-22T07:04:18.802058+0000","last_clean":"2024-04-22T07:04:18.802058+0000","last_became_active":"2024-04-22T07:04:12.788286+0000","last_became_peered":"2024-04-22T07:04:12.788286+0000","last_unstale":"2024-04-22T07:04:18.802058+0000","last_undegraded":"2024-04-22T07:04:18.802058+0000","last_fullsized":"2024-04-22T07:04:18.802058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961160+0000","last_change":"2024-04-22T07:04:13.804435+0000","last_active":"2024-04-22T07:04:14.961160+0000","last_peered":"2024-04-22T07:04:14.961160+0000","last_clean":"2024-04-22T07:04:14.961160+0000","last_became_active":"2024-04-22T07:04:13.804170+0000","last_became_peered":"2024-04-22T07:04:13.804170+0000","last_unstale":"2024-04-22T07:04:14.961160+0000","last_undegraded":"2024-04-22T07:04:14.961160+0000","last_fullsized":"2024-04-22T07:04:14.961160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.801438+0000","last_change":"2024-04-22T07:04:12.789905+0000","last_active":"2024-04-22T07:04:18.801438+0000","last_peered":"2024-04-22T07:04:18.801438+0000","last_clean":"2024-04-22T07:04:18.801438+0000","last_became_active":"2024-04-22T07:04:12.789503+0000","last_became_peered":"2024-04-22T07:04:12.789503+0000","last_unstale":"2024-04-22T07:04:18.801438+0000","last_undegraded":"2024-04-22T07:04:18.801438+0000","last_fullsized":"2024-04-22T07:04:18.801438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055741+0000","last_change":"2024-04-22T07:04:13.801490+0000","last_active":"2024-04-22T07:04:15.055741+0000","last_peered":"2024-04-22T07:04:15.055741+0000","last_clean":"2024-04-22T07:04:15.055741+0000","last_became_active":"2024-04-22T07:04:13.799356+0000","last_became_peered":"2024-04-22T07:04:13.799356+0000","last_unstale":"2024-04-22T07:04:15.055741+0000","last_undegraded":"2024-04-22T07:04:15.055741+0000","last_fullsized":"2024-04-22T07:04:15.055741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:32:04.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783198+0000","last_change":"2024-04-22T07:04:12.781089+0000","last_active":"2024-04-22T07:04:14.783198+0000","last_peered":"2024-04-22T07:04:14.783198+0000","last_clean":"2024-04-22T07:04:14.783198+0000","last_became_active":"2024-04-22T07:04:12.780827+0000","last_became_peered":"2024-04-22T07:04:12.780827+0000","last_unstale":"2024-04-22T07:04:14.783198+0000","last_undegraded":"2024-04-22T07:04:14.783198+0000","last_fullsized":"2024-04-22T07:04:14.783198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961202+0000","last_change":"2024-04-22T07:04:13.801375+0000","last_active":"2024-04-22T07:04:14.961202+0000","last_peered":"2024-04-22T07:04:14.961202+0000","last_clean":"2024-04-22T07:04:14.961202+0000","last_became_active":"2024-04-22T07:04:13.799488+0000","last_became_peered":"2024-04-22T07:04:13.799488+0000","last_unstale":"2024-04-22T07:04:14.961202+0000","last_undegraded":"2024-04-22T07:04:14.961202+0000","last_fullsized":"2024-04-22T07:04:14.961202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781576+0000","last_change":"2024-04-22T07:04:12.790758+0000","last_active":"2024-04-22T07:04:14.781576+0000","last_peered":"2024-04-22T07:04:14.781576+0000","last_clean":"2024-04-22T07:04:14.781576+0000","last_became_active":"2024-04-22T07:04:12.790444+0000","last_became_peered":"2024-04-22T07:04:12.790444+0000","last_unstale":"2024-04-22T07:04:14.781576+0000","last_undegraded":"2024-04-22T07:04:14.781576+0000","last_fullsized":"2024-04-22T07:04:14.781576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782297+0000","last_change":"2024-04-22T07:04:13.805415+0000","last_active":"2024-04-22T07:04:14.782297+0000","last_peered":"2024-04-22T07:04:14.782297+0000","last_clean":"2024-04-22T07:04:14.782297+0000","last_became_active":"2024-04-22T07:04:13.805115+0000","last_became_peered":"2024-04-22T07:04:13.805115+0000","last_unstale":"2024-04-22T07:04:14.782297+0000","last_undegraded":"2024-04-22T07:04:14.782297+0000","last_fullsized":"2024-04-22T07:04:14.782297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.803368+0000","last_change":"2024-04-22T07:04:12.789877+0000","last_active":"2024-04-22T07:04:18.803368+0000","last_peered":"2024-04-22T07:04:18.803368+0000","last_clean":"2024-04-22T07:04:18.803368+0000","last_became_active":"2024-04-22T07:04:12.789546+0000","last_became_peered":"2024-04-22T07:04:12.789546+0000","last_unstale":"2024-04-22T07:04:18.803368+0000","last_undegraded":"2024-04-22T07:04:18.803368+0000","last_fullsized":"2024-04-22T07:04:18.803368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:04.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961082+0000","last_change":"2024-04-22T07:04:13.801803+0000","last_active":"2024-04-22T07:04:14.961082+0000","last_peered":"2024-04-22T07:04:14.961082+0000","last_clean":"2024-04-22T07:04:14.961082+0000","last_became_active":"2024-04-22T07:04:13.801362+0000","last_became_peered":"2024-04-22T07:04:13.801362+0000","last_unstale":"2024-04-22T07:04:14.961082+0000","last_undegraded":"2024-04-22T07:04:14.961082+0000","last_fullsized":"2024-04-22T07:04:14.961082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.736119+0000","last_change":"2024-04-22T07:04:12.787917+0000","last_active":"2024-04-22T07:04:15.736119+0000","last_peered":"2024-04-22T07:04:15.736119+0000","last_clean":"2024-04-22T07:04:15.736119+0000","last_became_active":"2024-04-22T07:04:12.787636+0000","last_became_peered":"2024-04-22T07:04:12.787636+0000","last_unstale":"2024-04-22T07:04:15.736119+0000","last_undegraded":"2024-04-22T07:04:15.736119+0000","last_fullsized":"2024-04-22T07:04:15.736119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961456+0000","last_change":"2024-04-22T07:04:13.802786+0000","last_active":"2024-04-22T07:04:14.961456+0000","last_peered":"2024-04-22T07:04:14.961456+0000","last_clean":"2024-04-22T07:04:14.961456+0000","last_became_active":"2024-04-22T07:04:13.801299+0000","last_became_peered":"2024-04-22T07:04:13.801299+0000","last_unstale":"2024-04-22T07:04:14.961456+0000","last_undegraded":"2024-04-22T07:04:14.961456+0000","last_fullsized":"2024-04-22T07:04:14.961456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:28.362520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.800619+0000","last_change":"2024-04-22T07:04:12.788593+0000","last_active":"2024-04-22T07:04:18.800619+0000","last_peered":"2024-04-22T07:04:18.800619+0000","last_clean":"2024-04-22T07:04:18.800619+0000","last_became_active":"2024-04-22T07:04:12.788314+0000","last_became_peered":"2024-04-22T07:04:12.788314+0000","last_unstale":"2024-04-22T07:04:18.800619+0000","last_undegraded":"2024-04-22T07:04:18.800619+0000","last_fullsized":"2024-04-22T07:04:18.800619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:11:27.081266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961608+0000","last_change":"2024-04-22T07:04:13.800245+0000","last_active":"2024-04-22T07:04:14.961608+0000","last_peered":"2024-04-22T07:04:14.961608+0000","last_clean":"2024-04-22T07:04:14.961608+0000","last_became_active":"2024-04-22T07:04:13.799970+0000","last_became_peered":"2024-04-22T07:04:13.799970+0000","last_unstale":"2024-04-22T07:04:14.961608+0000","last_undegraded":"2024-04-22T07:04:14.961608+0000","last_fullsized":"2024-04-22T07:04:14.961608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:13.558356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.731834+0000","last_change":"2024-04-22T07:04:12.790003+0000","last_active":"2024-04-22T07:04:15.731834+0000","last_peered":"2024-04-22T07:04:15.731834+0000","last_clean":"2024-04-22T07:04:15.731834+0000","last_became_active":"2024-04-22T07:04:12.789708+0000","last_became_peered":"2024-04-22T07:04:12.789708+0000","last_unstale":"2024-04-22T07:04:15.731834+0000","last_undegraded":"2024-04-22T07:04:15.731834+0000","last_fullsized":"2024-04-22T07:04:15.731834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783345+0000","last_change":"2024-04-22T07:04:13.795082+0000","last_active":"2024-04-22T07:04:14.783345+0000","last_peered":"2024-04-22T07:04:14.783345+0000","last_clean":"2024-04-22T07:04:14.783345+0000","last_became_active":"2024-04-22T07:04:13.794793+0000","last_became_peered":"2024-04-22T07:04:13.794793+0000","last_unstale":"2024-04-22T07:04:14.783345+0000","last_undegraded":"2024-04-22T07:04:14.783345+0000","last_fullsized":"2024-04-22T07:04:14.783345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054608+0000","last_change":"2024-04-22T07:04:12.789252+0000","last_active":"2024-04-22T07:04:15.054608+0000","last_peered":"2024-04-22T07:04:15.054608+0000","last_clean":"2024-04-22T07:04:15.054608+0000","last_became_active":"2024-04-22T07:04:12.788946+0000","last_became_peered":"2024-04-22T07:04:12.788946+0000","last_unstale":"2024-04-22T07:04:15.054608+0000","last_undegraded":"2024-04-22T07:04:15.054608+0000","last_fullsized":"2024-04-22T07:04:15.054608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:36:13.428703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961272+0000","last_change":"2024-04-22T07:04:13.801250+0000","last_active":"2024-04-22T07:04:14.961272+0000","last_peered":"2024-04-22T07:04:14.961272+0000","last_clean":"2024-04-22T07:04:14.961272+0000","last_became_active":"2024-04-22T07:04:13.799143+0000","last_became_peered":"2024-04-22T07:04:13.799143+0000","last_unstale":"2024-04-22T07:04:14.961272+0000","last_undegraded":"2024-04-22T07:04:14.961272+0000","last_fullsized":"2024-04-22T07:04:14.961272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960952+0000","last_change":"2024-04-22T07:04:03.688462+0000","last_active":"2024-04-22T07:04:14.960952+0000","last_peered":"2024-04-22T07:04:14.960952+0000","last_clean":"2024-04-22T07:04:14.960952+0000","last_became_active":"2024-04-22T07:04:03.688191+0000","last_became_peered":"2024-04-22T07:04:03.688191+0000","last_unstale":"2024-04-22T07:04:14.960952+0000","last_undegraded":"2024-04-22T07:04:14.960952+0000","last_fullsized":"2024-04-22T07:04:14.960952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.464330+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.053999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782468+0000","last_change":"2024-04-22T07:04:12.783751+0000","last_active":"2024-04-22T07:04:14.782468+0000","last_peered":"2024-04-22T07:04:14.782468+0000","last_clean":"2024-04-22T07:04:14.782468+0000","last_became_active":"2024-04-22T07:04:12.783460+0000","last_became_peered":"2024-04-22T07:04:12.783460+0000","last_unstale":"2024-04-22T07:04:14.782468+0000","last_undegraded":"2024-04-22T07:04:14.782468+0000","last_fullsized":"2024-04-22T07:04:14.782468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054881+0000","last_change":"2024-04-22T07:04:13.791961+0000","last_active":"2024-04-22T07:04:15.054881+0000","last_peered":"2024-04-22T07:04:15.054881+0000","last_clean":"2024-04-22T07:04:15.054881+0000","last_became_active":"2024-04-22T07:04:13.790591+0000","last_became_peered":"2024-04-22T07:04:13.790591+0000","last_unstale":"2024-04-22T07:04:15.054881+0000","last_undegraded":"2024-04-22T07:04:15.054881+0000","last_fullsized":"2024-04-22T07:04:15.054881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960999+0000","last_change":"2024-04-22T07:04:12.787268+0000","last_active":"2024-04-22T07:04:14.960999+0000","last_peered":"2024-04-22T07:04:14.960999+0000","last_clean":"2024-04-22T07:04:14.960999+0000","last_became_active":"2024-04-22T07:04:12.786991+0000","last_became_peered":"2024-04-22T07:04:12.786991+0000","last_unstale":"2024-04-22T07:04:14.960999+0000","last_undegraded":"2024-04-22T07:04:14.960999+0000","last_fullsized":"2024-04-22T07:04:14.960999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054726+0000","last_change":"2024-04-22T07:04:13.800243+0000","last_active":"2024-04-22T07:04:15.054726+0000","last_peered":"2024-04-22T07:04:15.054726+0000","last_clean":"2024-04-22T07:04:15.054726+0000","last_became_active":"2024-04-22T07:04:13.799917+0000","last_became_peered":"2024-04-22T07:04:13.799917+0000","last_unstale":"2024-04-22T07:04:15.054726+0000","last_undegraded":"2024-04-22T07:04:15.054726+0000","last_fullsized":"2024-04-22T07:04:15.054726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:54.958916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054937+0000","last_change":"2024-04-22T07:04:12.787893+0000","last_active":"2024-04-22T07:04:15.054937+0000","last_peered":"2024-04-22T07:04:15.054937+0000","last_clean":"2024-04-22T07:04:15.054937+0000","last_became_active":"2024-04-22T07:04:12.787370+0000","last_became_peered":"2024-04-22T07:04:12.787370+0000","last_unstale":"2024-04-22T07:04:15.054937+0000","last_undegraded":"2024-04-22T07:04:15.054937+0000","last_fullsized":"2024-04-22T07:04:15.054937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:52:03.220670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055952+0000","last_change":"2024-04-22T07:04:13.795454+0000","last_active":"2024-04-22T07:04:15.055952+0000","last_peered":"2024-04-22T07:04:15.055952+0000","last_clean":"2024-04-22T07:04:15.055952+0000","last_became_active":"2024-04-22T07:04:13.795138+0000","last_became_peered":"2024-04-22T07:04:13.795138+0000","last_unstale":"2024-04-22T07:04:15.055952+0000","last_undegraded":"2024-04-22T07:04:15.055952+0000","last_fullsized":"2024-04-22T07:04:15.055952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783273+0000","last_change":"2024-04-22T07:04:12.786891+0000","last_active":"2024-04-22T07:04:14.783273+0000","last_peered":"2024-04-22T07:04:14.783273+0000","last_clean":"2024-04-22T07:04:14.783273+0000","last_became_active":"2024-04-22T07:04:12.786557+0000","last_became_peered":"2024-04-22T07:04:12.786557+0000","last_unstale":"2024-04-22T07:04:14.783273+0000","last_undegraded":"2024-04-22T07:04:14.783273+0000","last_fullsized":"2024-04-22T07:04:14.783273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055054+0000","last_change":"2024-04-22T07:04:13.798678+0000","last_active":"2024-04-22T07:04:15.055054+0000","last_peered":"2024-04-22T07:04:15.055054+0000","last_clean":"2024-04-22T07:04:15.055054+0000","last_became_active":"2024-04-22T07:04:13.798369+0000","last_became_peered":"2024-04-22T07:04:13.798369+0000","last_unstale":"2024-04-22T07:04:15.055054+0000","last_undegraded":"2024-04-22T07:04:15.055054+0000","last_fullsized":"2024-04-22T07:04:15.055054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:37.919431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055841+0000","last_change":"2024-04-22T07:04:12.791897+0000","last_active":"2024-04-22T07:04:15.055841+0000","last_peered":"2024-04-22T07:04:15.055841+0000","last_clean":"2024-04-22T07:04:15.055841+0000","last_became_active":"2024-04-22T07:04:12.791580+0000","last_became_peered":"2024-04-22T07:04:12.791580+0000","last_unstale":"2024-04-22T07:04:15.055841+0000","last_undegraded":"2024-04-22T07:04:15.055841+0000","last_fullsized":"2024-04-22T07:04:15.055841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782942+0000","last_change":"2024-04-22T07:04:13.803516+0000","last_active":"2024-04-22T07:04:14.782942+0000","last_peered":"2024-04-22T07:04:14.782942+0000","last_clean":"2024-04-22T07:04:14.782942+0000","last_became_active":"2024-04-22T07:04:13.803230+0000","last_became_peered":"2024-04-22T07:04:13.803230+0000","last_unstale":"2024-04-22T07:04:14.782942+0000","last_undegraded":"2024-04-22T07:04:14.782942+0000","last_fullsized":"2024-04-22T07:04:14.782942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961432+0000","last_change":"2024-04-22T07:04:12.789499+0000","last_active":"2024-04-22T07:04:14.961432+0000","last_peered":"2024-04-22T07:04:14.961432+0000","last_clean":"2024-04-22T07:04:14.961432+0000","last_became_active":"2024-04-22T07:04:12.789229+0000","last_became_peered":"2024-04-22T07:04:12.789229+0000","last_unstale":"2024-04-22T07:04:14.961432+0000","last_undegraded":"2024-04-22T07:04:14.961432+0000","last_fullsized":"2024-04-22T07:04:14.961432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:53.955113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055856+0000","last_change":"2024-04-22T07:04:13.797969+0000","last_active":"2024-04-22T07:04:15.055856+0000","last_peered":"2024-04-22T07:04:15.055856+0000","last_clean":"2024-04-22T07:04:15.055856+0000","last_became_active":"2024-04-22T07:04:13.797662+0000","last_became_peered":"2024-04-22T07:04:13.797662+0000","last_unstale":"2024-04-22T07:04:15.055856+0000","last_undegraded":"2024-04-22T07:04:15.055856+0000","last_fullsized":"2024-04-22T07:04:15.055856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056169+0000","last_change":"2024-04-22T07:04:12.791787+0000","last_active":"2024-04-22T07:04:15.056169+0000","last_peered":"2024-04-22T07:04:15.056169+0000","last_clean":"2024-04-22T07:04:15.056169+0000","last_became_active":"2024-04-22T07:04:12.791460+0000","last_became_peered":"2024-04-22T07:04:12.791460+0000","last_unstale":"2024-04-22T07:04:15.056169+0000","last_undegraded":"2024-04-22T07:04:15.056169+0000","last_fullsized":"2024-04-22T07:04:15.056169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055451+0000","last_change":"2024-04-22T07:04:13.797755+0000","last_active":"2024-04-22T07:04:15.055451+0000","last_peered":"2024-04-22T07:04:15.055451+0000","last_clean":"2024-04-22T07:04:15.055451+0000","last_became_active":"2024-04-22T07:04:13.797400+0000","last_became_peered":"2024-04-22T07:04:13.797400+0000","last_unstale":"2024-04-22T07:04:15.055451+0000","last_undegraded":"2024-04-22T07:04:15.055451+0000","last_fullsized":"2024-04-22T07:04:15.055451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.801465+0000","last_change":"2024-04-22T07:04:12.790458+0000","last_active":"2024-04-22T07:04:18.801465+0000","last_peered":"2024-04-22T07:04:18.801465+0000","last_clean":"2024-04-22T07:04:18.801465+0000","last_became_active":"2024-04-22T07:04:12.790209+0000","last_became_peered":"2024-04-22T07:04:12.790209+0000","last_unstale":"2024-04-22T07:04:18.801465+0000","last_undegraded":"2024-04-22T07:04:18.801465+0000","last_fullsized":"2024-04-22T07:04:18.801465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056058+0000","last_change":"2024-04-22T07:04:13.796176+0000","last_active":"2024-04-22T07:04:15.056058+0000","last_peered":"2024-04-22T07:04:15.056058+0000","last_clean":"2024-04-22T07:04:15.056058+0000","last_became_active":"2024-04-22T07:04:13.795325+0000","last_became_peered":"2024-04-22T07:04:13.795325+0000","last_unstale":"2024-04-22T07:04:15.056058+0000","last_undegraded":"2024-04-22T07:04:15.056058+0000","last_fullsized":"2024-04-22T07:04:15.056058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960859+0000","last_change":"2024-04-22T07:04:12.791157+0000","last_active":"2024-04-22T07:04:14.960859+0000","last_peered":"2024-04-22T07:04:14.960859+0000","last_clean":"2024-04-22T07:04:14.960859+0000","last_became_active":"2024-04-22T07:04:12.790879+0000","last_became_peered":"2024-04-22T07:04:12.790879+0000","last_unstale":"2024-04-22T07:04:14.960859+0000","last_undegraded":"2024-04-22T07:04:14.960859+0000","last_fullsized":"2024-04-22T07:04:14.960859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054928+0000","last_change":"2024-04-22T07:04:13.796587+0000","last_active":"2024-04-22T07:04:15.054928+0000","last_peered":"2024-04-22T07:04:15.054928+0000","last_clean":"2024-04-22T07:04:15.054928+0000","last_became_active":"2024-04-22T07:04:13.796305+0000","last_became_peered":"2024-04-22T07:04:13.796305+0000","last_unstale":"2024-04-22T07:04:15.054928+0000","last_undegraded":"2024-04-22T07:04:15.054928+0000","last_fullsized":"2024-04-22T07:04:15.054928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:29.793237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055857+0000","last_change":"2024-04-22T07:04:12.784359+0000","last_active":"2024-04-22T07:04:15.055857+0000","last_peered":"2024-04-22T07:04:15.055857+0000","last_clean":"2024-04-22T07:04:15.055857+0000","last_became_active":"2024-04-22T07:04:12.784076+0000","last_became_peered":"2024-04-22T07:04:12.784076+0000","last_unstale":"2024-04-22T07:04:15.055857+0000","last_undegraded":"2024-04-22T07:04:15.055857+0000","last_fullsized":"2024-04-22T07:04:15.055857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054875+0000","last_change":"2024-04-22T07:04:13.802658+0000","last_active":"2024-04-22T07:04:15.054875+0000","last_peered":"2024-04-22T07:04:15.054875+0000","last_clean":"2024-04-22T07:04:15.054875+0000","last_became_active":"2024-04-22T07:04:13.802059+0000","last_became_peered":"2024-04-22T07:04:13.802059+0000","last_unstale":"2024-04-22T07:04:15.054875+0000","last_undegraded":"2024-04-22T07:04:15.054875+0000","last_fullsized":"2024-04-22T07:04:15.054875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055829+0000","last_change":"2024-04-22T07:04:12.788090+0000","last_active":"2024-04-22T07:04:15.055829+0000","last_peered":"2024-04-22T07:04:15.055829+0000","last_clean":"2024-04-22T07:04:15.055829+0000","last_became_active":"2024-04-22T07:04:12.787818+0000","last_became_peered":"2024-04-22T07:04:12.787818+0000","last_unstale":"2024-04-22T07:04:15.055829+0000","last_undegraded":"2024-04-22T07:04:15.055829+0000","last_fullsized":"2024-04-22T07:04:15.055829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055442+0000","last_change":"2024-04-22T07:04:13.802013+0000","last_active":"2024-04-22T07:04:15.055442+0000","last_peered":"2024-04-22T07:04:15.055442+0000","last_clean":"2024-04-22T07:04:15.055442+0000","last_became_active":"2024-04-22T07:04:13.801404+0000","last_became_peered":"2024-04-22T07:04:13.801404+0000","last_unstale":"2024-04-22T07:04:15.055442+0000","last_undegraded":"2024-04-22T07:04:15.055442+0000","last_fullsized":"2024-04-22T07:04:15.055442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.801608+0000","last_change":"2024-04-22T07:04:12.790981+0000","last_active":"2024-04-22T07:04:18.801608+0000","last_peered":"2024-04-22T07:04:18.801608+0000","last_clean":"2024-04-22T07:04:18.801608+0000","last_became_active":"2024-04-22T07:04:12.790592+0000","last_became_peered":"2024-04-22T07:04:12.790592+0000","last_unstale":"2024-04-22T07:04:18.801608+0000","last_undegraded":"2024-04-22T07:04:18.801608+0000","last_fullsized":"2024-04-22T07:04:18.801608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:53.115405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781878+0000","last_change":"2024-04-22T07:04:13.800619+0000","last_active":"2024-04-22T07:04:14.781878+0000","last_peered":"2024-04-22T07:04:14.781878+0000","last_clean":"2024-04-22T07:04:14.781878+0000","last_became_active":"2024-04-22T07:04:13.800279+0000","last_became_peered":"2024-04-22T07:04:13.800279+0000","last_unstale":"2024-04-22T07:04:14.781878+0000","last_undegraded":"2024-04-22T07:04:14.781878+0000","last_fullsized":"2024-04-22T07:04:14.781878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782949+0000","last_change":"2024-04-22T07:04:13.789060+0000","last_active":"2024-04-22T07:04:14.782949+0000","last_peered":"2024-04-22T07:04:14.782949+0000","last_clean":"2024-04-22T07:04:14.782949+0000","last_became_active":"2024-04-22T07:04:13.788787+0000","last_became_peered":"2024-04-22T07:04:13.788787+0000","last_unstale":"2024-04-22T07:04:14.782949+0000","last_undegraded":"2024-04-22T07:04:14.782949+0000","last_fullsized":"2024-04-22T07:04:14.782949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.801044+0000","last_change":"2024-04-22T07:04:12.785077+0000","last_active":"2024-04-22T07:04:18.801044+0000","last_peered":"2024-04-22T07:04:18.801044+0000","last_clean":"2024-04-22T07:04:18.801044+0000","last_became_active":"2024-04-22T07:04:12.784804+0000","last_became_peered":"2024-04-22T07:04:12.784804+0000","last_unstale":"2024-04-22T07:04:18.801044+0000","last_undegraded":"2024-04-22T07:04:18.801044+0000","last_fullsized":"2024-04-22T07:04:18.801044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783259+0000","last_change":"2024-04-22T07:04:13.789866+0000","last_active":"2024-04-22T07:04:14.783259+0000","last_peered":"2024-04-22T07:04:14.783259+0000","last_clean":"2024-04-22T07:04:14.783259+0000","last_became_active":"2024-04-22T07:04:13.789539+0000","last_became_peered":"2024-04-22T07:04:13.789539+0000","last_unstale":"2024-04-22T07:04:14.783259+0000","last_undegraded":"2024-04-22T07:04:14.783259+0000","last_fullsized":"2024-04-22T07:04:14.783259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.801839+0000","last_change":"2024-04-22T07:04:12.786162+0000","last_active":"2024-04-22T07:04:18.801839+0000","last_peered":"2024-04-22T07:04:18.801839+0000","last_clean":"2024-04-22T07:04:18.801839+0000","last_became_active":"2024-04-22T07:04:12.785310+0000","last_became_peered":"2024-04-22T07:04:12.785310+0000","last_unstale":"2024-04-22T07:04:18.801839+0000","last_undegraded":"2024-04-22T07:04:18.801839+0000","last_fullsized":"2024-04-22T07:04:18.801839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:06.776205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056634+0000","last_change":"2024-04-22T07:04:13.796267+0000","last_active":"2024-04-22T07:04:15.056634+0000","last_peered":"2024-04-22T07:04:15.056634+0000","last_clean":"2024-04-22T07:04:15.056634+0000","last_became_active":"2024-04-22T07:04:13.795220+0000","last_became_peered":"2024-04-22T07:04:13.795220+0000","last_unstale":"2024-04-22T07:04:15.056634+0000","last_undegraded":"2024-04-22T07:04:15.056634+0000","last_fullsized":"2024-04-22T07:04:15.056634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.734368+0000","last_change":"2024-04-22T07:04:12.785962+0000","last_active":"2024-04-22T07:04:15.734368+0000","last_peered":"2024-04-22T07:04:15.734368+0000","last_clean":"2024-04-22T07:04:15.734368+0000","last_became_active":"2024-04-22T07:04:12.785608+0000","last_became_peered":"2024-04-22T07:04:12.785608+0000","last_unstale":"2024-04-22T07:04:15.734368+0000","last_undegraded":"2024-04-22T07:04:15.734368+0000","last_fullsized":"2024-04-22T07:04:15.734368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055613+0000","last_change":"2024-04-22T07:04:13.801662+0000","last_active":"2024-04-22T07:04:15.055613+0000","last_peered":"2024-04-22T07:04:15.055613+0000","last_clean":"2024-04-22T07:04:15.055613+0000","last_became_active":"2024-04-22T07:04:13.798282+0000","last_became_peered":"2024-04-22T07:04:13.798282+0000","last_unstale":"2024-04-22T07:04:15.055613+0000","last_undegraded":"2024-04-22T07:04:15.055613+0000","last_fullsized":"2024-04-22T07:04:15.055613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.802254+0000","last_change":"2024-04-22T07:04:12.791445+0000","last_active":"2024-04-22T07:04:18.802254+0000","last_peered":"2024-04-22T07:04:18.802254+0000","last_clean":"2024-04-22T07:04:18.802254+0000","last_became_active":"2024-04-22T07:04:12.791150+0000","last_became_peered":"2024-04-22T07:04:12.791150+0000","last_unstale":"2024-04-22T07:04:18.802254+0000","last_undegraded":"2024-04-22T07:04:18.802254+0000","last_fullsized":"2024-04-22T07:04:18.802254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":831488,"data_stored":672185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":81325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":245760,"data_stored":81361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":81395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":839680,"data_stored":672241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:22.748 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:22.851 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.853+0000 7fe0e40d2640 1 Processor -- start 2024-04-22T07:04:22.852 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.854+0000 7fe0e40d2640 1 -- start start 2024-04-22T07:04:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.854+0000 7fe0e40d2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0dc0c6d00 0x7fe0dc0c4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.854+0000 7fe0e40d2640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe0dc0c8a70 0x7fe0dc151850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.854+0000 7fe0e40d2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0dc0c6570 con 0x7fe0dc0c8690 2024-04-22T07:04:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.854+0000 7fe0e40d2640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fe0dc06d260 con 0x7fe0dc0c8a70 2024-04-22T07:04:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.854+0000 7fe0e40d2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fe0dc0c4eb0 con 0x7fe0dc0c6d00 2024-04-22T07:04:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.854+0000 7fe0e266f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe0dc0c8a70 0x7fe0dc151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.855+0000 7fe0e266f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe0dc0c8a70 0x7fe0dc151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38144/0 (socket says 172.21.15.44:38144) 2024-04-22T07:04:22.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.855+0000 7fe0e266f640 1 -- 172.21.15.44:0/3016538910 learned_addr learned my addr 172.21.15.44:0/3016538910 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:22.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.855+0000 7fe0e266f640 1 -- 172.21.15.44:0/3016538910 >> v2:172.21.15.70:3300/0 conn(0x7fe0dc0c6d00 msgr2=0x7fe0dc0c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:22.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.855+0000 7fe0e266f640 1 --2- 172.21.15.44:0/3016538910 >> v2:172.21.15.70:3300/0 conn(0x7fe0dc0c6d00 0x7fe0dc0c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.855+0000 7fe0e266f640 1 -- 172.21.15.44:0/3016538910 >> v1:172.21.15.70:6789/0 conn(0x7fe0dc0c8690 legacy=0x7fe0dc0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:22.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.855+0000 7fe0e266f640 1 -- 172.21.15.44:0/3016538910 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc0c9420 con 0x7fe0dc0c8a70 2024-04-22T07:04:22.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.855+0000 7fe0e266f640 1 --2- 172.21.15.44:0/3016538910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8a70 0x7fe0dc151850 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0c9d80 tx=0x7fe0d802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5e6c461ab578bc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e0e6c640 1 -- 172.21.15.44:0/3016538910 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d802ca80 con 0x7fe0dc0c8a70 2024-04-22T07:04:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e0e6c640 1 -- 172.21.15.44:0/3016538910 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d802cc20 con 0x7fe0dc0c8a70 2024-04-22T07:04:22.858 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e0e6c640 1 -- 172.21.15.44:0/3016538910 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80343f0 con 0x7fe0dc0c8a70 2024-04-22T07:04:22.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3016538910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8a70 msgr2=0x7fe0dc151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e40d2640 1 --2- 172.21.15.44:0/3016538910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8a70 0x7fe0dc151850 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0c9d80 tx=0x7fe0d802bee0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3016538910 shutdown_connections 2024-04-22T07:04:22.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e40d2640 1 --2- 172.21.15.44:0/3016538910 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8a70 0x7fe0dc151850 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e40d2640 1 --2- 172.21.15.44:0/3016538910 >> v2:172.21.15.70:3300/0 conn(0x7fe0dc0c6d00 0x7fe0dc0c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.859 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3016538910 >> 172.21.15.44:0/3016538910 conn(0x7fe0dc0c9e20 msgr2=0x7fe0dc0ca200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:22.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3016538910 shutdown_connections 2024-04-22T07:04:22.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.856+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3016538910 wait complete. 2024-04-22T07:04:22.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.857+0000 7fe0e40d2640 1 Processor -- start 2024-04-22T07:04:22.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.857+0000 7fe0e40d2640 1 -- start start 2024-04-22T07:04:22.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e40d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0dc0c6d00 0x7fe0dc141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.860 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e40d2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8690 0x7fe0dc145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e40d2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0dc0c8a70 0x7fe0dc145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e40d2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc0c7d20 con 0x7fe0dc0c8690 2024-04-22T07:04:22.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e40d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc0c99e0 con 0x7fe0dc0c6d00 2024-04-22T07:04:22.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e40d2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe0dc0c7ea0 con 0x7fe0dc0c8a70 2024-04-22T07:04:22.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e166d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8690 0x7fe0dc145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.861 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e166d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8690 0x7fe0dc145950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38152/0 (socket says 172.21.15.44:38152) 2024-04-22T07:04:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e166d640 1 -- 172.21.15.44:0/3683293278 learned_addr learned my addr 172.21.15.44:0/3683293278 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e266f640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0dc0c8a70 0x7fe0dc145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.858+0000 7fe0e1e6e640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0dc0c6d00 0x7fe0dc141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0e166d640 1 -- 172.21.15.44:0/3683293278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0dc0c8a70 msgr2=0x7fe0dc145e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0e166d640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0dc0c8a70 0x7fe0dc145e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.862 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0e166d640 1 -- 172.21.15.44:0/3683293278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0dc0c6d00 msgr2=0x7fe0dc141690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0e166d640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0dc0c6d00 0x7fe0dc141690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0e166d640 1 -- 172.21.15.44:0/3683293278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc1f1e50 con 0x7fe0dc0c8690 2024-04-22T07:04:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0e1e6e640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0dc0c6d00 0x7fe0dc141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0e166d640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8690 0x7fe0dc145950 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00b410 tx=0x7fe0cc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0d27fc640 1 -- 172.21.15.44:0/3683293278 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc015020 con 0x7fe0dc0c8690 2024-04-22T07:04:22.863 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0d27fc640 1 -- 172.21.15.44:0/3683293278 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0cc011830 con 0x7fe0dc0c8690 2024-04-22T07:04:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3683293278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc1408d0 con 0x7fe0dc0c8690 2024-04-22T07:04:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0d27fc640 1 -- 172.21.15.44:0/3683293278 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc011c20 con 0x7fe0dc0c8690 2024-04-22T07:04:22.864 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.859+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3683293278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc140e90 con 0x7fe0dc0c8690 2024-04-22T07:04:22.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.860+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3683293278 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0dc0c8ee0 con 0x7fe0dc0c8690 2024-04-22T07:04:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.860+0000 7fe0d27fc640 1 -- 172.21.15.44:0/3683293278 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0cc018020 con 0x7fe0dc0c8690 2024-04-22T07:04:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.861+0000 7fe0d27fc640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe0c006c8f0 0x7fe0c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.861+0000 7fe0d27fc640 1 -- 172.21.15.44:0/3683293278 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe0cc08ba80 con 0x7fe0dc0c8690 2024-04-22T07:04:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.861+0000 7fe0e1e6e640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe0c006c8f0 0x7fe0c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.861+0000 7fe0e1e6e640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe0c006c8f0 0x7fe0c006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400a810 tx=0x7fe0d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:22.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:22.865+0000 7fe0d27fc640 1 -- 172.21.15.44:0/3683293278 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe0dc0c8ee0 con 0x7fe0dc0c8690 2024-04-22T07:04:23.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.045+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3683293278 --> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0dc141bd0 con 0x7fe0c006c8f0 2024-04-22T07:04:23.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.051+0000 7fe0d27fc640 1 -- 172.21.15.44:0/3683293278 <== mgr.4101 v2:172.21.15.44:6832/1889438723 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251859 (secure 0 0 0) 0x7fe0d400ad90 con 0x7fe0c006c8f0 2024-04-22T07:04:23.051 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:23.053 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T07:04:23.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.054+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3683293278 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe0c006c8f0 msgr2=0x7fe0c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:23.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.054+0000 7fe0e40d2640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe0c006c8f0 0x7fe0c006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400a810 tx=0x7fe0d4008040 comp rx=0 tx=0).stop 2024-04-22T07:04:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.055+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3683293278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8690 msgr2=0x7fe0dc145950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.055+0000 7fe0e40d2640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8690 0x7fe0dc145950 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00b410 tx=0x7fe0cc00b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.055+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3683293278 shutdown_connections 2024-04-22T07:04:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.055+0000 7fe0e40d2640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe0c006c8f0 0x7fe0c006edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.055+0000 7fe0e40d2640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe0dc0c8a70 0x7fe0dc145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.055+0000 7fe0e40d2640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe0dc0c8690 0x7fe0dc145950 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.055+0000 7fe0e40d2640 1 --2- 172.21.15.44:0/3683293278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0dc0c6d00 0x7fe0dc141690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.055+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3683293278 >> 172.21.15.44:0/3683293278 conn(0x7fe0dc0c9e20 msgr2=0x7fe0dc0706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.055+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3683293278 shutdown_connections 2024-04-22T07:04:23.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.055+0000 7fe0e40d2640 1 -- 172.21.15.44:0/3683293278 wait complete. 2024-04-22T07:04:23.063 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-04-22T07:04:21.713176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239976,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734744,"statfs":{"total":773094113280,"available":772848377856,"internally_reserved":0,"allocated":3121152,"data_stored":1835436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003167"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961498+0000","last_change":"2024-04-22T07:04:13.800864+0000","last_active":"2024-04-22T07:04:14.961498+0000","last_peered":"2024-04-22T07:04:14.961498+0000","last_clean":"2024-04-22T07:04:14.961498+0000","last_became_active":"2024-04-22T07:04:13.800611+0000","last_became_peered":"2024-04-22T07:04:13.800611+0000","last_unstale":"2024-04-22T07:04:14.961498+0000","last_undegraded":"2024-04-22T07:04:14.961498+0000","last_fullsized":"2024-04-22T07:04:14.961498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:55.129127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056077+0000","last_change":"2024-04-22T07:04:13.798390+0000","last_active":"2024-04-22T07:04:15.056077+0000","last_peered":"2024-04-22T07:04:15.056077+0000","last_clean":"2024-04-22T07:04:15.056077+0000","last_became_active":"2024-04-22T07:04:13.797927+0000","last_became_peered":"2024-04-22T07:04:13.797927+0000","last_unstale":"2024-04-22T07:04:15.056077+0000","last_undegraded":"2024-04-22T07:04:15.056077+0000","last_fullsized":"2024-04-22T07:04:15.056077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056425+0000","last_change":"2024-04-22T07:04:13.795288+0000","last_active":"2024-04-22T07:04:15.056425+0000","last_peered":"2024-04-22T07:04:15.056425+0000","last_clean":"2024-04-22T07:04:15.056425+0000","last_became_active":"2024-04-22T07:04:13.794704+0000","last_became_peered":"2024-04-22T07:04:13.794704+0000","last_unstale":"2024-04-22T07:04:15.056425+0000","last_undegraded":"2024-04-22T07:04:15.056425+0000","last_fullsized":"2024-04-22T07:04:15.056425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783223+0000","last_change":"2024-04-22T07:04:13.801884+0000","last_active":"2024-04-22T07:04:14.783223+0000","last_peered":"2024-04-22T07:04:14.783223+0000","last_clean":"2024-04-22T07:04:14.783223+0000","last_became_active":"2024-04-22T07:04:13.801620+0000","last_became_peered":"2024-04-22T07:04:13.801620+0000","last_unstale":"2024-04-22T07:04:14.783223+0000","last_undegraded":"2024-04-22T07:04:14.783223+0000","last_fullsized":"2024-04-22T07:04:14.783223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961151+0000","last_change":"2024-04-22T07:04:13.798670+0000","last_active":"2024-04-22T07:04:14.961151+0000","last_peered":"2024-04-22T07:04:14.961151+0000","last_clean":"2024-04-22T07:04:14.961151+0000","last_became_active":"2024-04-22T07:04:13.798384+0000","last_became_peered":"2024-04-22T07:04:13.798384+0000","last_unstale":"2024-04-22T07:04:14.961151+0000","last_undegraded":"2024-04-22T07:04:14.961151+0000","last_fullsized":"2024-04-22T07:04:14.961151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783124+0000","last_change":"2024-04-22T07:04:13.796468+0000","last_active":"2024-04-22T07:04:14.783124+0000","last_peered":"2024-04-22T07:04:14.783124+0000","last_clean":"2024-04-22T07:04:14.783124+0000","last_became_active":"2024-04-22T07:04:13.796139+0000","last_became_peered":"2024-04-22T07:04:13.796139+0000","last_unstale":"2024-04-22T07:04:14.783124+0000","last_undegraded":"2024-04-22T07:04:14.783124+0000","last_fullsized":"2024-04-22T07:04:14.783124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961066+0000","last_change":"2024-04-22T07:04:13.801449+0000","last_active":"2024-04-22T07:04:14.961066+0000","last_peered":"2024-04-22T07:04:14.961066+0000","last_clean":"2024-04-22T07:04:14.961066+0000","last_became_active":"2024-04-22T07:04:13.799374+0000","last_became_peered":"2024-04-22T07:04:13.799374+0000","last_unstale":"2024-04-22T07:04:14.961066+0000","last_undegraded":"2024-04-22T07:04:14.961066+0000","last_fullsized":"2024-04-22T07:04:14.961066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054559+0000","last_change":"2024-04-22T07:04:13.802780+0000","last_active":"2024-04-22T07:04:15.054559+0000","last_peered":"2024-04-22T07:04:15.054559+0000","last_clean":"2024-04-22T07:04:15.054559+0000","last_became_active":"2024-04-22T07:04:13.802371+0000","last_became_peered":"2024-04-22T07:04:13.802371+0000","last_unstale":"2024-04-22T07:04:15.054559+0000","last_undegraded":"2024-04-22T07:04:15.054559+0000","last_fullsized":"2024-04-22T07:04:15.054559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961532+0000","last_change":"2024-04-22T07:04:13.792483+0000","last_active":"2024-04-22T07:04:14.961532+0000","last_peered":"2024-04-22T07:04:14.961532+0000","last_clean":"2024-04-22T07:04:14.961532+0000","last_became_active":"2024-04-22T07:04:13.792198+0000","last_became_peered":"2024-04-22T07:04:13.792198+0000","last_unstale":"2024-04-22T07:04:14.961532+0000","last_undegraded":"2024-04-22T07:04:14.961532+0000","last_fullsized":"2024-04-22T07:04:14.961532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:36.943428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782040+0000","last_change":"2024-04-22T07:04:13.797379+0000","last_active":"2024-04-22T07:04:14.782040+0000","last_peered":"2024-04-22T07:04:14.782040+0000","last_clean":"2024-04-22T07:04:14.782040+0000","last_became_active":"2024-04-22T07:04:13.797029+0000","last_became_peered":"2024-04-22T07:04:13.797029+0000","last_unstale":"2024-04-22T07:04:14.782040+0000","last_undegraded":"2024-04-22T07:04:14.782040+0000","last_fullsized":"2024-04-22T07:04:14.782040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781680+0000","last_change":"2024-04-22T07:04:13.789875+0000","last_active":"2024-04-22T07:04:14.781680+0000","last_peered":"2024-04-22T07:04:14.781680+0000","last_clean":"2024-04-22T07:04:14.781680+0000","last_became_active":"2024-04-22T07:04:13.789598+0000","last_became_peered":"2024-04-22T07:04:13.789598+0000","last_unstale":"2024-04-22T07:04:14.781680+0000","last_undegraded":"2024-04-22T07:04:14.781680+0000","last_fullsized":"2024-04-22T07:04:14.781680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055156+0000","last_change":"2024-04-22T07:04:13.791103+0000","last_active":"2024-04-22T07:04:15.055156+0000","last_peered":"2024-04-22T07:04:15.055156+0000","last_clean":"2024-04-22T07:04:15.055156+0000","last_became_active":"2024-04-22T07:04:13.790819+0000","last_became_peered":"2024-04-22T07:04:13.790819+0000","last_unstale":"2024-04-22T07:04:15.055156+0000","last_undegraded":"2024-04-22T07:04:15.055156+0000","last_fullsized":"2024-04-22T07:04:15.055156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:50.632020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055402+0000","last_change":"2024-04-22T07:04:13.802407+0000","last_active":"2024-04-22T07:04:15.055402+0000","last_peered":"2024-04-22T07:04:15.055402+0000","last_clean":"2024-04-22T07:04:15.055402+0000","last_became_active":"2024-04-22T07:04:13.801785+0000","last_became_peered":"2024-04-22T07:04:13.801785+0000","last_unstale":"2024-04-22T07:04:15.055402+0000","last_undegraded":"2024-04-22T07:04:15.055402+0000","last_fullsized":"2024-04-22T07:04:15.055402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055557+0000","last_change":"2024-04-22T07:04:13.795888+0000","last_active":"2024-04-22T07:04:15.055557+0000","last_peered":"2024-04-22T07:04:15.055557+0000","last_clean":"2024-04-22T07:04:15.055557+0000","last_became_active":"2024-04-22T07:04:13.795092+0000","last_became_peered":"2024-04-22T07:04:13.795092+0000","last_unstale":"2024-04-22T07:04:15.055557+0000","last_undegraded":"2024-04-22T07:04:15.055557+0000","last_fullsized":"2024-04-22T07:04:15.055557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:54.119208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055926+0000","last_change":"2024-04-22T07:04:13.797138+0000","last_active":"2024-04-22T07:04:15.055926+0000","last_peered":"2024-04-22T07:04:15.055926+0000","last_clean":"2024-04-22T07:04:15.055926+0000","last_became_active":"2024-04-22T07:04:13.796763+0000","last_became_peered":"2024-04-22T07:04:13.796763+0000","last_unstale":"2024-04-22T07:04:15.055926+0000","last_undegraded":"2024-04-22T07:04:15.055926+0000","last_fullsized":"2024-04-22T07:04:15.055926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782465+0000","last_change":"2024-04-22T07:04:13.803869+0000","last_active":"2024-04-22T07:04:14.782465+0000","last_peered":"2024-04-22T07:04:14.782465+0000","last_clean":"2024-04-22T07:04:14.782465+0000","last_became_active":"2024-04-22T07:04:13.803451+0000","last_became_peered":"2024-04-22T07:04:13.803451+0000","last_unstale":"2024-04-22T07:04:14.782465+0000","last_undegraded":"2024-04-22T07:04:14.782465+0000","last_fullsized":"2024-04-22T07:04:14.782465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783077+0000","last_change":"2024-04-22T07:04:13.795746+0000","last_active":"2024-04-22T07:04:14.783077+0000","last_peered":"2024-04-22T07:04:14.783077+0000","last_clean":"2024-04-22T07:04:14.783077+0000","last_became_active":"2024-04-22T07:04:13.794694+0000","last_became_peered":"2024-04-22T07:04:13.794694+0000","last_unstale":"2024-04-22T07:04:14.783077+0000","last_undegraded":"2024-04-22T07:04:14.783077+0000","last_fullsized":"2024-04-22T07:04:14.783077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054829+0000","last_change":"2024-04-22T07:04:13.790670+0000","last_active":"2024-04-22T07:04:15.054829+0000","last_peered":"2024-04-22T07:04:15.054829+0000","last_clean":"2024-04-22T07:04:15.054829+0000","last_became_active":"2024-04-22T07:04:13.790104+0000","last_became_peered":"2024-04-22T07:04:13.790104+0000","last_unstale":"2024-04-22T07:04:15.054829+0000","last_undegraded":"2024-04-22T07:04:15.054829+0000","last_fullsized":"2024-04-22T07:04:15.054829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:09.325971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961464+0000","last_change":"2024-04-22T07:04:13.799016+0000","last_active":"2024-04-22T07:04:14.961464+0000","last_peered":"2024-04-22T07:04:14.961464+0000","last_clean":"2024-04-22T07:04:14.961464+0000","last_became_active":"2024-04-22T07:04:13.798716+0000","last_became_peered":"2024-04-22T07:04:13.798716+0000","last_unstale":"2024-04-22T07:04:14.961464+0000","last_undegraded":"2024-04-22T07:04:14.961464+0000","last_fullsized":"2024-04-22T07:04:14.961464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:36.609382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781586+0000","last_change":"2024-04-22T07:04:13.797296+0000","last_active":"2024-04-22T07:04:14.781586+0000","last_peered":"2024-04-22T07:04:14.781586+0000","last_clean":"2024-04-22T07:04:14.781586+0000","last_became_active":"2024-04-22T07:04:13.796997+0000","last_became_peered":"2024-04-22T07:04:13.796997+0000","last_unstale":"2024-04-22T07:04:14.781586+0000","last_undegraded":"2024-04-22T07:04:14.781586+0000","last_fullsized":"2024-04-22T07:04:14.781586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055994+0000","last_change":"2024-04-22T07:04:13.799675+0000","last_active":"2024-04-22T07:04:15.055994+0000","last_peered":"2024-04-22T07:04:15.055994+0000","last_clean":"2024-04-22T07:04:15.055994+0000","last_became_active":"2024-04-22T07:04:13.799411+0000","last_became_peered":"2024-04-22T07:04:13.799411+0000","last_unstale":"2024-04-22T07:04:15.055994+0000","last_undegraded":"2024-04-22T07:04:15.055994+0000","last_fullsized":"2024-04-22T07:04:15.055994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783058+0000","last_change":"2024-04-22T07:04:13.796128+0000","last_active":"2024-04-22T07:04:14.783058+0000","last_peered":"2024-04-22T07:04:14.783058+0000","last_clean":"2024-04-22T07:04:14.783058+0000","last_became_active":"2024-04-22T07:04:13.795330+0000","last_became_peered":"2024-04-22T07:04:13.795330+0000","last_unstale":"2024-04-22T07:04:14.783058+0000","last_undegraded":"2024-04-22T07:04:14.783058+0000","last_fullsized":"2024-04-22T07:04:14.783058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055294+0000","last_change":"2024-04-22T07:04:13.800458+0000","last_active":"2024-04-22T07:04:15.055294+0000","last_peered":"2024-04-22T07:04:15.055294+0000","last_clean":"2024-04-22T07:04:15.055294+0000","last_became_active":"2024-04-22T07:04:13.800171+0000","last_became_peered":"2024-04-22T07:04:13.800171+0000","last_unstale":"2024-04-22T07:04:15.055294+0000","last_undegraded":"2024-04-22T07:04:15.055294+0000","last_fullsized":"2024-04-22T07:04:15.055294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782518+0000","last_change":"2024-04-22T07:04:13.802472+0000","last_active":"2024-04-22T07:04:14.782518+0000","last_peered":"2024-04-22T07:04:14.782518+0000","last_clean":"2024-04-22T07:04:14.782518+0000","last_became_active":"2024-04-22T07:04:13.802201+0000","last_became_peered":"2024-04-22T07:04:13.802201+0000","last_unstale":"2024-04-22T07:04:14.782518+0000","last_undegraded":"2024-04-22T07:04:14.782518+0000","last_fullsized":"2024-04-22T07:04:14.782518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055335+0000","last_change":"2024-04-22T07:04:13.795055+0000","last_active":"2024-04-22T07:04:15.055335+0000","last_peered":"2024-04-22T07:04:15.055335+0000","last_clean":"2024-04-22T07:04:15.055335+0000","last_became_active":"2024-04-22T07:04:13.794145+0000","last_became_peered":"2024-04-22T07:04:13.794145+0000","last_unstale":"2024-04-22T07:04:15.055335+0000","last_undegraded":"2024-04-22T07:04:15.055335+0000","last_fullsized":"2024-04-22T07:04:15.055335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782104+0000","last_change":"2024-04-22T07:04:13.799772+0000","last_active":"2024-04-22T07:04:14.782104+0000","last_peered":"2024-04-22T07:04:14.782104+0000","last_clean":"2024-04-22T07:04:14.782104+0000","last_became_active":"2024-04-22T07:04:13.799504+0000","last_became_peered":"2024-04-22T07:04:13.799504+0000","last_unstale":"2024-04-22T07:04:14.782104+0000","last_undegraded":"2024-04-22T07:04:14.782104+0000","last_fullsized":"2024-04-22T07:04:14.782104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056133+0000","last_change":"2024-04-22T07:04:13.804604+0000","last_active":"2024-04-22T07:04:15.056133+0000","last_peered":"2024-04-22T07:04:15.056133+0000","last_clean":"2024-04-22T07:04:15.056133+0000","last_became_active":"2024-04-22T07:04:13.803594+0000","last_became_peered":"2024-04-22T07:04:13.803594+0000","last_unstale":"2024-04-22T07:04:15.056133+0000","last_undegraded":"2024-04-22T07:04:15.056133+0000","last_fullsized":"2024-04-22T07:04:15.056133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055262+0000","last_change":"2024-04-22T07:04:13.795947+0000","last_active":"2024-04-22T07:04:15.055262+0000","last_peered":"2024-04-22T07:04:15.055262+0000","last_clean":"2024-04-22T07:04:15.055262+0000","last_became_active":"2024-04-22T07:04:13.795223+0000","last_became_peered":"2024-04-22T07:04:13.795223+0000","last_unstale":"2024-04-22T07:04:15.055262+0000","last_undegraded":"2024-04-22T07:04:15.055262+0000","last_fullsized":"2024-04-22T07:04:15.055262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056195+0000","last_change":"2024-04-22T07:04:13.804225+0000","last_active":"2024-04-22T07:04:15.056195+0000","last_peered":"2024-04-22T07:04:15.056195+0000","last_clean":"2024-04-22T07:04:15.056195+0000","last_became_active":"2024-04-22T07:04:13.803920+0000","last_became_peered":"2024-04-22T07:04:13.803920+0000","last_unstale":"2024-04-22T07:04:15.056195+0000","last_undegraded":"2024-04-22T07:04:15.056195+0000","last_fullsized":"2024-04-22T07:04:15.056195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783228+0000","last_change":"2024-04-22T07:04:13.796357+0000","last_active":"2024-04-22T07:04:14.783228+0000","last_peered":"2024-04-22T07:04:14.783228+0000","last_clean":"2024-04-22T07:04:14.783228+0000","last_became_active":"2024-04-22T07:04:13.795784+0000","last_became_peered":"2024-04-22T07:04:13.795784+0000","last_unstale":"2024-04-22T07:04:14.783228+0000","last_undegraded":"2024-04-22T07:04:14.783228+0000","last_fullsized":"2024-04-22T07:04:14.783228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054855+0000","last_change":"2024-04-22T07:04:13.784337+0000","last_active":"2024-04-22T07:04:15.054855+0000","last_peered":"2024-04-22T07:04:15.054855+0000","last_clean":"2024-04-22T07:04:15.054855+0000","last_became_active":"2024-04-22T07:04:13.783914+0000","last_became_peered":"2024-04-22T07:04:13.783914+0000","last_unstale":"2024-04-22T07:04:15.054855+0000","last_undegraded":"2024-04-22T07:04:15.054855+0000","last_fullsized":"2024-04-22T07:04:15.054855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:43:41.211395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781515+0000","last_change":"2024-04-22T07:04:13.806969+0000","last_active":"2024-04-22T07:04:14.781515+0000","last_peered":"2024-04-22T07:04:14.781515+0000","last_clean":"2024-04-22T07:04:14.781515+0000","last_became_active":"2024-04-22T07:04:13.806715+0000","last_became_peered":"2024-04-22T07:04:13.806715+0000","last_unstale":"2024-04-22T07:04:14.781515+0000","last_undegraded":"2024-04-22T07:04:14.781515+0000","last_fullsized":"2024-04-22T07:04:14.781515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054895+0000","last_change":"2024-04-22T07:04:13.794896+0000","last_active":"2024-04-22T07:04:15.054895+0000","last_peered":"2024-04-22T07:04:15.054895+0000","last_clean":"2024-04-22T07:04:15.054895+0000","last_became_active":"2024-04-22T07:04:13.794605+0000","last_became_peered":"2024-04-22T07:04:13.794605+0000","last_unstale":"2024-04-22T07:04:15.054895+0000","last_undegraded":"2024-04-22T07:04:15.054895+0000","last_fullsized":"2024-04-22T07:04:15.054895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:02.613130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.734312+0000","last_change":"2024-04-22T07:04:12.786738+0000","last_active":"2024-04-22T07:04:15.734312+0000","last_peered":"2024-04-22T07:04:15.734312+0000","last_clean":"2024-04-22T07:04:15.734312+0000","last_became_active":"2024-04-22T07:04:12.786413+0000","last_became_peered":"2024-04-22T07:04:12.786413+0000","last_unstale":"2024-04-22T07:04:15.734312+0000","last_undegraded":"2024-04-22T07:04:15.734312+0000","last_fullsized":"2024-04-22T07:04:15.734312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056196+0000","last_change":"2024-04-22T07:04:13.800525+0000","last_active":"2024-04-22T07:04:15.056196+0000","last_peered":"2024-04-22T07:04:15.056196+0000","last_clean":"2024-04-22T07:04:15.056196+0000","last_became_active":"2024-04-22T07:04:13.800254+0000","last_became_peered":"2024-04-22T07:04:13.800254+0000","last_unstale":"2024-04-22T07:04:15.056196+0000","last_undegraded":"2024-04-22T07:04:15.056196+0000","last_fullsized":"2024-04-22T07:04:15.056196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.733258+0000","last_change":"2024-04-22T07:04:12.786726+0000","last_active":"2024-04-22T07:04:15.733258+0000","last_peered":"2024-04-22T07:04:15.733258+0000","last_clean":"2024-04-22T07:04:15.733258+0000","last_became_active":"2024-04-22T07:04:12.786444+0000","last_became_peered":"2024-04-22T07:04:12.786444+0000","last_unstale":"2024-04-22T07:04:15.733258+0000","last_undegraded":"2024-04-22T07:04:15.733258+0000","last_fullsized":"2024-04-22T07:04:15.733258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055394+0000","last_change":"2024-04-22T07:04:12.787357+0000","last_active":"2024-04-22T07:04:15.055394+0000","last_peered":"2024-04-22T07:04:15.055394+0000","last_clean":"2024-04-22T07:04:15.055394+0000","last_became_active":"2024-04-22T07:04:12.787066+0000","last_became_peered":"2024-04-22T07:04:12.787066+0000","last_unstale":"2024-04-22T07:04:15.055394+0000","last_undegraded":"2024-04-22T07:04:15.055394+0000","last_fullsized":"2024-04-22T07:04:15.055394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054713+0000","last_change":"2024-04-22T07:04:13.807697+0000","last_active":"2024-04-22T07:04:15.054713+0000","last_peered":"2024-04-22T07:04:15.054713+0000","last_clean":"2024-04-22T07:04:15.054713+0000","last_became_active":"2024-04-22T07:04:13.807327+0000","last_became_peered":"2024-04-22T07:04:13.807327+0000","last_unstale":"2024-04-22T07:04:15.054713+0000","last_undegraded":"2024-04-22T07:04:15.054713+0000","last_fullsized":"2024-04-22T07:04:15.054713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783313+0000","last_change":"2024-04-22T07:04:12.787783+0000","last_active":"2024-04-22T07:04:14.783313+0000","last_peered":"2024-04-22T07:04:14.783313+0000","last_clean":"2024-04-22T07:04:14.783313+0000","last_became_active":"2024-04-22T07:04:12.786765+0000","last_became_peered":"2024-04-22T07:04:12.786765+0000","last_unstale":"2024-04-22T07:04:14.783313+0000","last_undegraded":"2024-04-22T07:04:14.783313+0000","last_fullsized":"2024-04-22T07:04:14.783313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055740+0000","last_change":"2024-04-22T07:04:13.804931+0000","last_active":"2024-04-22T07:04:15.055740+0000","last_peered":"2024-04-22T07:04:15.055740+0000","last_clean":"2024-04-22T07:04:15.055740+0000","last_became_active":"2024-04-22T07:04:13.804629+0000","last_became_peered":"2024-04-22T07:04:13.804629+0000","last_unstale":"2024-04-22T07:04:15.055740+0000","last_undegraded":"2024-04-22T07:04:15.055740+0000","last_fullsized":"2024-04-22T07:04:15.055740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781808+0000","last_change":"2024-04-22T07:04:12.789279+0000","last_active":"2024-04-22T07:04:14.781808+0000","last_peered":"2024-04-22T07:04:14.781808+0000","last_clean":"2024-04-22T07:04:14.781808+0000","last_became_active":"2024-04-22T07:04:12.787705+0000","last_became_peered":"2024-04-22T07:04:12.787705+0000","last_unstale":"2024-04-22T07:04:14.781808+0000","last_undegraded":"2024-04-22T07:04:14.781808+0000","last_fullsized":"2024-04-22T07:04:14.781808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782812+0000","last_change":"2024-04-22T07:04:13.794292+0000","last_active":"2024-04-22T07:04:14.782812+0000","last_peered":"2024-04-22T07:04:14.782812+0000","last_clean":"2024-04-22T07:04:14.782812+0000","last_became_active":"2024-04-22T07:04:13.793986+0000","last_became_peered":"2024-04-22T07:04:13.793986+0000","last_unstale":"2024-04-22T07:04:14.782812+0000","last_undegraded":"2024-04-22T07:04:14.782812+0000","last_fullsized":"2024-04-22T07:04:14.782812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782990+0000","last_change":"2024-04-22T07:04:12.787064+0000","last_active":"2024-04-22T07:04:14.782990+0000","last_peered":"2024-04-22T07:04:14.782990+0000","last_clean":"2024-04-22T07:04:14.782990+0000","last_became_active":"2024-04-22T07:04:12.786430+0000","last_became_peered":"2024-04-22T07:04:12.786430+0000","last_unstale":"2024-04-22T07:04:14.782990+0000","last_undegraded":"2024-04-22T07:04:14.782990+0000","last_fullsized":"2024-04-22T07:04:14.782990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056732+0000","last_change":"2024-04-22T07:04:13.801337+0000","last_active":"2024-04-22T07:04:15.056732+0000","last_peered":"2024-04-22T07:04:15.056732+0000","last_clean":"2024-04-22T07:04:15.056732+0000","last_became_active":"2024-04-22T07:04:13.800916+0000","last_became_peered":"2024-04-22T07:04:13.800916+0000","last_unstale":"2024-04-22T07:04:15.056732+0000","last_undegraded":"2024-04-22T07:04:15.056732+0000","last_fullsized":"2024-04-22T07:04:15.056732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.803158+0000","last_change":"2024-04-22T07:04:12.787741+0000","last_active":"2024-04-22T07:04:18.803158+0000","last_peered":"2024-04-22T07:04:18.803158+0000","last_clean":"2024-04-22T07:04:18.803158+0000","last_became_active":"2024-04-22T07:04:12.787405+0000","last_became_peered":"2024-04-22T07:04:12.787405+0000","last_unstale":"2024-04-22T07:04:18.803158+0000","last_undegraded":"2024-04-22T07:04:18.803158+0000","last_fullsized":"2024-04-22T07:04:18.803158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055184+0000","last_change":"2024-04-22T07:04:13.795502+0000","last_active":"2024-04-22T07:04:15.055184+0000","last_peered":"2024-04-22T07:04:15.055184+0000","last_clean":"2024-04-22T07:04:15.055184+0000","last_became_active":"2024-04-22T07:04:13.795199+0000","last_became_peered":"2024-04-22T07:04:13.795199+0000","last_unstale":"2024-04-22T07:04:15.055184+0000","last_undegraded":"2024-04-22T07:04:15.055184+0000","last_fullsized":"2024-04-22T07:04:15.055184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:07.780008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.802058+0000","last_change":"2024-04-22T07:04:12.788572+0000","last_active":"2024-04-22T07:04:18.802058+0000","last_peered":"2024-04-22T07:04:18.802058+0000","last_clean":"2024-04-22T07:04:18.802058+0000","last_became_active":"2024-04-22T07:04:12.788286+0000","last_became_peered":"2024-04-22T07:04:12.788286+0000","last_unstale":"2024-04-22T07:04:18.802058+0000","last_undegraded":"2024-04-22T07:04:18.802058+0000","last_fullsized":"2024-04-22T07:04:18.802058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961160+0000","last_change":"2024-04-22T07:04:13.804435+0000","last_active":"2024-04-22T07:04:14.961160+0000","last_peered":"2024-04-22T07:04:14.961160+0000","last_clean":"2024-04-22T07:04:14.961160+0000","last_became_active":"2024-04-22T07:04:13.804170+0000","last_became_peered":"2024-04-22T07:04:13.804170+0000","last_unstale":"2024-04-22T07:04:14.961160+0000","last_undegraded":"2024-04-22T07:04:14.961160+0000","last_fullsized":"2024-04-22T07:04:14.961160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.801438+0000","last_change":"2024-04-22T07:04:12.789905+0000","last_active":"2024-04-22T07:04:18.801438+0000","last_peered":"2024-04-22T07:04:18.801438+0000","last_clean":"2024-04-22T07:04:18.801438+0000","last_became_active":"2024-04-22T07:04:12.789503+0000","last_became_peered":"2024-04-22T07:04:12.789503+0000","last_unstale":"2024-04-22T07:04:18.801438+0000","last_undegraded":"2024-04-22T07:04:18.801438+0000","last_fullsized":"2024-04-22T07:04:18.801438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055741+0000","last_change":"2024-04-22T07:04:13.801490+0000","last_active":"2024-04-22T07:04:15.055741+0000","last_peered":"2024-04-22T07:04:15.055741+0000","last_clean":"2024-04-22T07:04:15.055741+0000","last_became_active":"2024-04-22T07:04:13.799356+0000","last_became_peered":"2024-04-22T07:04:13.799356+0000","last_unstale":"2024-04-22T07:04:15.055741+0000","last_undegraded":"2024-04-22T07:04:15.055741+0000","last_fullsized":"2024-04-22T07:04:15.055741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:32:04.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783198+0000","last_change":"2024-04-22T07:04:12.781089+0000","last_active":"2024-04-22T07:04:14.783198+0000","last_peered":"2024-04-22T07:04:14.783198+0000","last_clean":"2024-04-22T07:04:14.783198+0000","last_became_active":"2024-04-22T07:04:12.780827+0000","last_became_peered":"2024-04-22T07:04:12.780827+0000","last_unstale":"2024-04-22T07:04:14.783198+0000","last_undegraded":"2024-04-22T07:04:14.783198+0000","last_fullsized":"2024-04-22T07:04:14.783198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961202+0000","last_change":"2024-04-22T07:04:13.801375+0000","last_active":"2024-04-22T07:04:14.961202+0000","last_peered":"2024-04-22T07:04:14.961202+0000","last_clean":"2024-04-22T07:04:14.961202+0000","last_became_active":"2024-04-22T07:04:13.799488+0000","last_became_peered":"2024-04-22T07:04:13.799488+0000","last_unstale":"2024-04-22T07:04:14.961202+0000","last_undegraded":"2024-04-22T07:04:14.961202+0000","last_fullsized":"2024-04-22T07:04:14.961202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781576+0000","last_change":"2024-04-22T07:04:12.790758+0000","last_active":"2024-04-22T07:04:14.781576+0000","last_peered":"2024-04-22T07:04:14.781576+0000","last_clean":"2024-04-22T07:04:14.781576+0000","last_became_active":"2024-04-22T07:04:12.790444+0000","last_became_peered":"2024-04-22T07:04:12.790444+0000","last_unstale":"2024-04-22T07:04:14.781576+0000","last_undegraded":"2024-04-22T07:04:14.781576+0000","last_fullsized":"2024-04-22T07:04:14.781576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782297+0000","last_change":"2024-04-22T07:04:13.805415+0000","last_active":"2024-04-22T07:04:14.782297+0000","last_peered":"2024-04-22T07:04:14.782297+0000","last_clean":"2024-04-22T07:04:14.782297+0000","last_became_active":"2024-04-22T07:04:13.805115+0000","last_became_peered":"2024-04-22T07:04:13.805115+0000","last_unstale":"2024-04-22T07:04:14.782297+0000","last_undegraded":"2024-04-22T07:04:14.782297+0000","last_fullsized":"2024-04-22T07:04:14.782297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.803368+0000","last_change":"2024-04-22T07:04:12.789877+0000","last_active":"2024-04-22T07:04:18.803368+0000","last_peered":"2024-04-22T07:04:18.803368+0000","last_clean":"2024-04-22T07:04:18.803368+0000","last_became_active":"2024-04-22T07:04:12.789546+0000","last_became_peered":"2024-04-22T07:04:12.789546+0000","last_unstale":"2024-04-22T07:04:18.803368+0000","last_undegraded":"2024-04-22T07:04:18.803368+0000","last_fullsized":"2024-04-22T07:04:18.803368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:04.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961082+0000","last_change":"2024-04-22T07:04:13.801803+0000","last_active":"2024-04-22T07:04:14.961082+0000","last_peered":"2024-04-22T07:04:14.961082+0000","last_clean":"2024-04-22T07:04:14.961082+0000","last_became_active":"2024-04-22T07:04:13.801362+0000","last_became_peered":"2024-04-22T07:04:13.801362+0000","last_unstale":"2024-04-22T07:04:14.961082+0000","last_undegraded":"2024-04-22T07:04:14.961082+0000","last_fullsized":"2024-04-22T07:04:14.961082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.736119+0000","last_change":"2024-04-22T07:04:12.787917+0000","last_active":"2024-04-22T07:04:15.736119+0000","last_peered":"2024-04-22T07:04:15.736119+0000","last_clean":"2024-04-22T07:04:15.736119+0000","last_became_active":"2024-04-22T07:04:12.787636+0000","last_became_peered":"2024-04-22T07:04:12.787636+0000","last_unstale":"2024-04-22T07:04:15.736119+0000","last_undegraded":"2024-04-22T07:04:15.736119+0000","last_fullsized":"2024-04-22T07:04:15.736119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961456+0000","last_change":"2024-04-22T07:04:13.802786+0000","last_active":"2024-04-22T07:04:14.961456+0000","last_peered":"2024-04-22T07:04:14.961456+0000","last_clean":"2024-04-22T07:04:14.961456+0000","last_became_active":"2024-04-22T07:04:13.801299+0000","last_became_peered":"2024-04-22T07:04:13.801299+0000","last_unstale":"2024-04-22T07:04:14.961456+0000","last_undegraded":"2024-04-22T07:04:14.961456+0000","last_fullsized":"2024-04-22T07:04:14.961456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:28.362520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.800619+0000","last_change":"2024-04-22T07:04:12.788593+0000","last_active":"2024-04-22T07:04:18.800619+0000","last_peered":"2024-04-22T07:04:18.800619+0000","last_clean":"2024-04-22T07:04:18.800619+0000","last_became_active":"2024-04-22T07:04:12.788314+0000","last_became_peered":"2024-04-22T07:04:12.788314+0000","last_unstale":"2024-04-22T07:04:18.800619+0000","last_undegraded":"2024-04-22T07:04:18.800619+0000","last_fullsized":"2024-04-22T07:04:18.800619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:11:27.081266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961608+0000","last_change":"2024-04-22T07:04:13.800245+0000","last_active":"2024-04-22T07:04:14.961608+0000","last_peered":"2024-04-22T07:04:14.961608+0000","last_clean":"2024-04-22T07:04:14.961608+0000","last_became_active":"2024-04-22T07:04:13.799970+0000","last_became_peered":"2024-04-22T07:04:13.799970+0000","last_unstale":"2024-04-22T07:04:14.961608+0000","last_undegraded":"2024-04-22T07:04:14.961608+0000","last_fullsized":"2024-04-22T07:04:14.961608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:13.558356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.731834+0000","last_change":"2024-04-22T07:04:12.790003+0000","last_active":"2024-04-22T07:04:15.731834+0000","last_peered":"2024-04-22T07:04:15.731834+0000","last_clean":"2024-04-22T07:04:15.731834+0000","last_became_active":"2024-04-22T07:04:12.789708+0000","last_became_peered":"2024-04-22T07:04:12.789708+0000","last_unstale":"2024-04-22T07:04:15.731834+0000","last_undegraded":"2024-04-22T07:04:15.731834+0000","last_fullsized":"2024-04-22T07:04:15.731834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783345+0000","last_change":"2024-04-22T07:04:13.795082+0000","last_active":"2024-04-22T07:04:14.783345+0000","last_peered":"2024-04-22T07:04:14.783345+0000","last_clean":"2024-04-22T07:04:14.783345+0000","last_became_active":"2024-04-22T07:04:13.794793+0000","last_became_peered":"2024-04-22T07:04:13.794793+0000","last_unstale":"2024-04-22T07:04:14.783345+0000","last_undegraded":"2024-04-22T07:04:14.783345+0000","last_fullsized":"2024-04-22T07:04:14.783345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054608+0000","last_change":"2024-04-22T07:04:12.789252+0000","last_active":"2024-04-22T07:04:15.054608+0000","last_peered":"2024-04-22T07:04:15.054608+0000","last_clean":"2024-04-22T07:04:15.054608+0000","last_became_active":"2024-04-22T07:04:12.788946+0000","last_became_peered":"2024-04-22T07:04:12.788946+0000","last_unstale":"2024-04-22T07:04:15.054608+0000","last_undegraded":"2024-04-22T07:04:15.054608+0000","last_fullsized":"2024-04-22T07:04:15.054608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:36:13.428703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961272+0000","last_change":"2024-04-22T07:04:13.801250+0000","last_active":"2024-04-22T07:04:14.961272+0000","last_peered":"2024-04-22T07:04:14.961272+0000","last_clean":"2024-04-22T07:04:14.961272+0000","last_became_active":"2024-04-22T07:04:13.799143+0000","last_became_peered":"2024-04-22T07:04:13.799143+0000","last_unstale":"2024-04-22T07:04:14.961272+0000","last_undegraded":"2024-04-22T07:04:14.961272+0000","last_fullsized":"2024-04-22T07:04:14.961272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960952+0000","last_change":"2024-04-22T07:04:03.688462+0000","last_active":"2024-04-22T07:04:14.960952+0000","last_peered":"2024-04-22T07:04:14.960952+0000","last_clean":"2024-04-22T07:04:14.960952+0000","last_became_active":"2024-04-22T07:04:03.688191+0000","last_became_peered":"2024-04-22T07:04:03.688191+0000","last_unstale":"2024-04-22T07:04:14.960952+0000","last_undegraded":"2024-04-22T07:04:14.960952+0000","last_fullsized":"2024-04-22T07:04:14.960952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.464330+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.053999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782468+0000","last_change":"2024-04-22T07:04:12.783751+0000","last_active":"2024-04-22T07:04:14.782468+0000","last_peered":"2024-04-22T07:04:14.782468+0000","last_clean":"2024-04-22T07:04:14.782468+0000","last_became_active":"2024-04-22T07:04:12.783460+0000","last_became_peered":"2024-04-22T07:04:12.783460+0000","last_unstale":"2024-04-22T07:04:14.782468+0000","last_undegraded":"2024-04-22T07:04:14.782468+0000","last_fullsized":"2024-04-22T07:04:14.782468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054881+0000","last_change":"2024-04-22T07:04:13.791961+0000","last_active":"2024-04-22T07:04:15.054881+0000","last_peered":"2024-04-22T07:04:15.054881+0000","last_clean":"2024-04-22T07:04:15.054881+0000","last_became_active":"2024-04-22T07:04:13.790591+0000","last_became_peered":"2024-04-22T07:04:13.790591+0000","last_unstale":"2024-04-22T07:04:15.054881+0000","last_undegraded":"2024-04-22T07:04:15.054881+0000","last_fullsized":"2024-04-22T07:04:15.054881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960999+0000","last_change":"2024-04-22T07:04:12.787268+0000","last_active":"2024-04-22T07:04:14.960999+0000","last_peered":"2024-04-22T07:04:14.960999+0000","last_clean":"2024-04-22T07:04:14.960999+0000","last_became_active":"2024-04-22T07:04:12.786991+0000","last_became_peered":"2024-04-22T07:04:12.786991+0000","last_unstale":"2024-04-22T07:04:14.960999+0000","last_undegraded":"2024-04-22T07:04:14.960999+0000","last_fullsized":"2024-04-22T07:04:14.960999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054726+0000","last_change":"2024-04-22T07:04:13.800243+0000","last_active":"2024-04-22T07:04:15.054726+0000","last_peered":"2024-04-22T07:04:15.054726+0000","last_clean":"2024-04-22T07:04:15.054726+0000","last_became_active":"2024-04-22T07:04:13.799917+0000","last_became_peered":"2024-04-22T07:04:13.799917+0000","last_unstale":"2024-04-22T07:04:15.054726+0000","last_undegraded":"2024-04-22T07:04:15.054726+0000","last_fullsized":"2024-04-22T07:04:15.054726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:54.958916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054937+0000","last_change":"2024-04-22T07:04:12.787893+0000","last_active":"2024-04-22T07:04:15.054937+0000","last_peered":"2024-04-22T07:04:15.054937+0000","last_clean":"2024-04-22T07:04:15.054937+0000","last_became_active":"2024-04-22T07:04:12.787370+0000","last_became_peered":"2024-04-22T07:04:12.787370+0000","last_unstale":"2024-04-22T07:04:15.054937+0000","last_undegraded":"2024-04-22T07:04:15.054937+0000","last_fullsized":"2024-04-22T07:04:15.054937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:52:03.220670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055952+0000","last_change":"2024-04-22T07:04:13.795454+0000","last_active":"2024-04-22T07:04:15.055952+0000","last_peered":"2024-04-22T07:04:15.055952+0000","last_clean":"2024-04-22T07:04:15.055952+0000","last_became_active":"2024-04-22T07:04:13.795138+0000","last_became_peered":"2024-04-22T07:04:13.795138+0000","last_unstale":"2024-04-22T07:04:15.055952+0000","last_undegraded":"2024-04-22T07:04:15.055952+0000","last_fullsized":"2024-04-22T07:04:15.055952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783273+0000","last_change":"2024-04-22T07:04:12.786891+0000","last_active":"2024-04-22T07:04:14.783273+0000","last_peered":"2024-04-22T07:04:14.783273+0000","last_clean":"2024-04-22T07:04:14.783273+0000","last_became_active":"2024-04-22T07:04:12.786557+0000","last_became_peered":"2024-04-22T07:04:12.786557+0000","last_unstale":"2024-04-22T07:04:14.783273+0000","last_undegraded":"2024-04-22T07:04:14.783273+0000","last_fullsized":"2024-04-22T07:04:14.783273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055054+0000","last_change":"2024-04-22T07:04:13.798678+0000","last_active":"2024-04-22T07:04:15.055054+0000","last_peered":"2024-04-22T07:04:15.055054+0000","last_clean":"2024-04-22T07:04:15.055054+0000","last_became_active":"2024-04-22T07:04:13.798369+0000","last_became_peered":"2024-04-22T07:04:13.798369+0000","last_unstale":"2024-04-22T07:04:15.055054+0000","last_undegraded":"2024-04-22T07:04:15.055054+0000","last_fullsized":"2024-04-22T07:04:15.055054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:37.919431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055841+0000","last_change":"2024-04-22T07:04:12.791897+0000","last_active":"2024-04-22T07:04:15.055841+0000","last_peered":"2024-04-22T07:04:15.055841+0000","last_clean":"2024-04-22T07:04:15.055841+0000","last_became_active":"2024-04-22T07:04:12.791580+0000","last_became_peered":"2024-04-22T07:04:12.791580+0000","last_unstale":"2024-04-22T07:04:15.055841+0000","last_undegraded":"2024-04-22T07:04:15.055841+0000","last_fullsized":"2024-04-22T07:04:15.055841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782942+0000","last_change":"2024-04-22T07:04:13.803516+0000","last_active":"2024-04-22T07:04:14.782942+0000","last_peered":"2024-04-22T07:04:14.782942+0000","last_clean":"2024-04-22T07:04:14.782942+0000","last_became_active":"2024-04-22T07:04:13.803230+0000","last_became_peered":"2024-04-22T07:04:13.803230+0000","last_unstale":"2024-04-22T07:04:14.782942+0000","last_undegraded":"2024-04-22T07:04:14.782942+0000","last_fullsized":"2024-04-22T07:04:14.782942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961432+0000","last_change":"2024-04-22T07:04:12.789499+0000","last_active":"2024-04-22T07:04:14.961432+0000","last_peered":"2024-04-22T07:04:14.961432+0000","last_clean":"2024-04-22T07:04:14.961432+0000","last_became_active":"2024-04-22T07:04:12.789229+0000","last_became_peered":"2024-04-22T07:04:12.789229+0000","last_unstale":"2024-04-22T07:04:14.961432+0000","last_undegraded":"2024-04-22T07:04:14.961432+0000","last_fullsized":"2024-04-22T07:04:14.961432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:53.955113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055856+0000","last_change":"2024-04-22T07:04:13.797969+0000","last_active":"2024-04-22T07:04:15.055856+0000","last_peered":"2024-04-22T07:04:15.055856+0000","last_clean":"2024-04-22T07:04:15.055856+0000","last_became_active":"2024-04-22T07:04:13.797662+0000","last_became_peered":"2024-04-22T07:04:13.797662+0000","last_unstale":"2024-04-22T07:04:15.055856+0000","last_undegraded":"2024-04-22T07:04:15.055856+0000","last_fullsized":"2024-04-22T07:04:15.055856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056169+0000","last_change":"2024-04-22T07:04:12.791787+0000","last_active":"2024-04-22T07:04:15.056169+0000","last_peered":"2024-04-22T07:04:15.056169+0000","last_clean":"2024-04-22T07:04:15.056169+0000","last_became_active":"2024-04-22T07:04:12.791460+0000","last_became_peered":"2024-04-22T07:04:12.791460+0000","last_unstale":"2024-04-22T07:04:15.056169+0000","last_undegraded":"2024-04-22T07:04:15.056169+0000","last_fullsized":"2024-04-22T07:04:15.056169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055451+0000","last_change":"2024-04-22T07:04:13.797755+0000","last_active":"2024-04-22T07:04:15.055451+0000","last_peered":"2024-04-22T07:04:15.055451+0000","last_clean":"2024-04-22T07:04:15.055451+0000","last_became_active":"2024-04-22T07:04:13.797400+0000","last_became_peered":"2024-04-22T07:04:13.797400+0000","last_unstale":"2024-04-22T07:04:15.055451+0000","last_undegraded":"2024-04-22T07:04:15.055451+0000","last_fullsized":"2024-04-22T07:04:15.055451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.801465+0000","last_change":"2024-04-22T07:04:12.790458+0000","last_active":"2024-04-22T07:04:18.801465+0000","last_peered":"2024-04-22T07:04:18.801465+0000","last_clean":"2024-04-22T07:04:18.801465+0000","last_became_active":"2024-04-22T07:04:12.790209+0000","last_became_peered":"2024-04-22T07:04:12.790209+0000","last_unstale":"2024-04-22T07:04:18.801465+0000","last_undegraded":"2024-04-22T07:04:18.801465+0000","last_fullsized":"2024-04-22T07:04:18.801465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056058+0000","last_change":"2024-04-22T07:04:13.796176+0000","last_active":"2024-04-22T07:04:15.056058+0000","last_peered":"2024-04-22T07:04:15.056058+0000","last_clean":"2024-04-22T07:04:15.056058+0000","last_became_active":"2024-04-22T07:04:13.795325+0000","last_became_peered":"2024-04-22T07:04:13.795325+0000","last_unstale":"2024-04-22T07:04:15.056058+0000","last_undegraded":"2024-04-22T07:04:15.056058+0000","last_fullsized":"2024-04-22T07:04:15.056058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960859+0000","last_change":"2024-04-22T07:04:12.791157+0000","last_active":"2024-04-22T07:04:14.960859+0000","last_peered":"2024-04-22T07:04:14.960859+0000","last_clean":"2024-04-22T07:04:14.960859+0000","last_became_active":"2024-04-22T07:04:12.790879+0000","last_became_peered":"2024-04-22T07:04:12.790879+0000","last_unstale":"2024-04-22T07:04:14.960859+0000","last_undegraded":"2024-04-22T07:04:14.960859+0000","last_fullsized":"2024-04-22T07:04:14.960859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054928+0000","last_change":"2024-04-22T07:04:13.796587+0000","last_active":"2024-04-22T07:04:15.054928+0000","last_peered":"2024-04-22T07:04:15.054928+0000","last_clean":"2024-04-22T07:04:15.054928+0000","last_became_active":"2024-04-22T07:04:13.796305+0000","last_became_peered":"2024-04-22T07:04:13.796305+0000","last_unstale":"2024-04-22T07:04:15.054928+0000","last_undegraded":"2024-04-22T07:04:15.054928+0000","last_fullsized":"2024-04-22T07:04:15.054928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:29.793237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055857+0000","last_change":"2024-04-22T07:04:12.784359+0000","last_active":"2024-04-22T07:04:15.055857+0000","last_peered":"2024-04-22T07:04:15.055857+0000","last_clean":"2024-04-22T07:04:15.055857+0000","last_became_active":"2024-04-22T07:04:12.784076+0000","last_became_peered":"2024-04-22T07:04:12.784076+0000","last_unstale":"2024-04-22T07:04:15.055857+0000","last_undegraded":"2024-04-22T07:04:15.055857+0000","last_fullsized":"2024-04-22T07:04:15.055857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054875+0000","last_change":"2024-04-22T07:04:13.802658+0000","last_active":"2024-04-22T07:04:15.054875+0000","last_peered":"2024-04-22T07:04:15.054875+0000","last_clean":"2024-04-22T07:04:15.054875+0000","last_became_active":"2024-04-22T07:04:13.802059+0000","last_became_peered":"2024-04-22T07:04:13.802059+0000","last_unstale":"2024-04-22T07:04:15.054875+0000","last_undegraded":"2024-04-22T07:04:15.054875+0000","last_fullsized":"2024-04-22T07:04:15.054875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055829+0000","last_change":"2024-04-22T07:04:12.788090+0000","last_active":"2024-04-22T07:04:15.055829+0000","last_peered":"2024-04-22T07:04:15.055829+0000","last_clean":"2024-04-22T07:04:15.055829+0000","last_became_active":"2024-04-22T07:04:12.787818+0000","last_became_peered":"2024-04-22T07:04:12.787818+0000","last_unstale":"2024-04-22T07:04:15.055829+0000","last_undegraded":"2024-04-22T07:04:15.055829+0000","last_fullsized":"2024-04-22T07:04:15.055829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055442+0000","last_change":"2024-04-22T07:04:13.802013+0000","last_active":"2024-04-22T07:04:15.055442+0000","last_peered":"2024-04-22T07:04:15.055442+0000","last_clean":"2024-04-22T07:04:15.055442+0000","last_became_active":"2024-04-22T07:04:13.801404+0000","last_became_peered":"2024-04-22T07:04:13.801404+0000","last_unstale":"2024-04-22T07:04:15.055442+0000","last_undegraded":"2024-04-22T07:04:15.055442+0000","last_fullsized":"2024-04-22T07:04:15.055442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.801608+0000","last_change":"2024-04-22T07:04:12.790981+0000","last_active":"2024-04-22T07:04:18.801608+0000","last_peered":"2024-04-22T07:04:18.801608+0000","last_clean":"2024-04-22T07:04:18.801608+0000","last_became_active":"2024-04-22T07:04:12.790592+0000","last_became_peered":"2024-04-22T07:04:12.790592+0000","last_unstale":"2024-04-22T07:04:18.801608+0000","last_undegraded":"2024-04-22T07:04:18.801608+0000","last_fullsized":"2024-04-22T07:04:18.801608+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:53.115405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781878+0000","last_change":"2024-04-22T07:04:13.800619+0000","last_active":"2024-04-22T07:04:14.781878+0000","last_peered":"2024-04-22T07:04:14.781878+0000","last_clean":"2024-04-22T07:04:14.781878+0000","last_became_active":"2024-04-22T07:04:13.800279+0000","last_became_peered":"2024-04-22T07:04:13.800279+0000","last_unstale":"2024-04-22T07:04:14.781878+0000","last_undegraded":"2024-04-22T07:04:14.781878+0000","last_fullsized":"2024-04-22T07:04:14.781878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782949+0000","last_change":"2024-04-22T07:04:13.789060+0000","last_active":"2024-04-22T07:04:14.782949+0000","last_peered":"2024-04-22T07:04:14.782949+0000","last_clean":"2024-04-22T07:04:14.782949+0000","last_became_active":"2024-04-22T07:04:13.788787+0000","last_became_peered":"2024-04-22T07:04:13.788787+0000","last_unstale":"2024-04-22T07:04:14.782949+0000","last_undegraded":"2024-04-22T07:04:14.782949+0000","last_fullsized":"2024-04-22T07:04:14.782949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.801044+0000","last_change":"2024-04-22T07:04:12.785077+0000","last_active":"2024-04-22T07:04:18.801044+0000","last_peered":"2024-04-22T07:04:18.801044+0000","last_clean":"2024-04-22T07:04:18.801044+0000","last_became_active":"2024-04-22T07:04:12.784804+0000","last_became_peered":"2024-04-22T07:04:12.784804+0000","last_unstale":"2024-04-22T07:04:18.801044+0000","last_undegraded":"2024-04-22T07:04:18.801044+0000","last_fullsized":"2024-04-22T07:04:18.801044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783259+0000","last_change":"2024-04-22T07:04:13.789866+0000","last_active":"2024-04-22T07:04:14.783259+0000","last_peered":"2024-04-22T07:04:14.783259+0000","last_clean":"2024-04-22T07:04:14.783259+0000","last_became_active":"2024-04-22T07:04:13.789539+0000","last_became_peered":"2024-04-22T07:04:13.789539+0000","last_unstale":"2024-04-22T07:04:14.783259+0000","last_undegraded":"2024-04-22T07:04:14.783259+0000","last_fullsized":"2024-04-22T07:04:14.783259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.801839+0000","last_change":"2024-04-22T07:04:12.786162+0000","last_active":"2024-04-22T07:04:18.801839+0000","last_peered":"2024-04-22T07:04:18.801839+0000","last_clean":"2024-04-22T07:04:18.801839+0000","last_became_active":"2024-04-22T07:04:12.785310+0000","last_became_peered":"2024-04-22T07:04:12.785310+0000","last_unstale":"2024-04-22T07:04:18.801839+0000","last_undegraded":"2024-04-22T07:04:18.801839+0000","last_fullsized":"2024-04-22T07:04:18.801839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:06.776205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056634+0000","last_change":"2024-04-22T07:04:13.796267+0000","last_active":"2024-04-22T07:04:15.056634+0000","last_peered":"2024-04-22T07:04:15.056634+0000","last_clean":"2024-04-22T07:04:15.056634+0000","last_became_active":"2024-04-22T07:04:13.795220+0000","last_became_peered":"2024-04-22T07:04:13.795220+0000","last_unstale":"2024-04-22T07:04:15.056634+0000","last_undegraded":"2024-04-22T07:04:15.056634+0000","last_fullsized":"2024-04-22T07:04:15.056634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.734368+0000","last_change":"2024-04-22T07:04:12.785962+0000","last_active":"2024-04-22T07:04:15.734368+0000","last_peered":"2024-04-22T07:04:15.734368+0000","last_clean":"2024-04-22T07:04:15.734368+0000","last_became_active":"2024-04-22T07:04:12.785608+0000","last_became_peered":"2024-04-22T07:04:12.785608+0000","last_unstale":"2024-04-22T07:04:15.734368+0000","last_undegraded":"2024-04-22T07:04:15.734368+0000","last_fullsized":"2024-04-22T07:04:15.734368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055613+0000","last_change":"2024-04-22T07:04:13.801662+0000","last_active":"2024-04-22T07:04:15.055613+0000","last_peered":"2024-04-22T07:04:15.055613+0000","last_clean":"2024-04-22T07:04:15.055613+0000","last_became_active":"2024-04-22T07:04:13.798282+0000","last_became_peered":"2024-04-22T07:04:13.798282+0000","last_unstale":"2024-04-22T07:04:15.055613+0000","last_undegraded":"2024-04-22T07:04:15.055613+0000","last_fullsized":"2024-04-22T07:04:15.055613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.802254+0000","last_change":"2024-04-22T07:04:12.791445+0000","last_active":"2024-04-22T07:04:18.802254+0000","last_peered":"2024-04-22T07:04:18.802254+0000","last_clean":"2024-04-22T07:04:18.802254+0000","last_became_active":"2024-04-22T07:04:12.791150+0000","last_became_peered":"2024-04-22T07:04:12.791150+0000","last_unstale":"2024-04-22T07:04:18.802254+0000","last_undegraded":"2024-04-22T07:04:18.802254+0000","last_fullsized":"2024-04-22T07:04:18.802254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":831488,"data_stored":672185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":237568,"data_stored":81325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":245760,"data_stored":81361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":82847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":81395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":839680,"data_stored":672241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":233472,"data_stored":81235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:23.069 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-22T07:04:23.069 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-22T07:04:23.069 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-22T07:04:23.069 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T07:04:23.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.207+0000 7f5ad433d640 1 Processor -- start 2024-04-22T07:04:23.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.207+0000 7f5ad433d640 1 -- start start 2024-04-22T07:04:23.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad433d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5acc1550a0 0x7f5acc0ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:23.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad433d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5acc15b6f0 0x7f5acc0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:23.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad433d640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5acc0c9710 con 0x7f5acc0c9df0 2024-04-22T07:04:23.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad433d640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f5acc06fc40 con 0x7f5acc1550a0 2024-04-22T07:04:23.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad433d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5acc0c99f0 con 0x7f5acc15b6f0 2024-04-22T07:04:23.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad20d9640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5acc1550a0 0x7f5acc0ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:23.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad18d8640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f5acc0c9df0 0x7f5acc15afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38112/0 (socket says 172.21.15.44:38112) 2024-04-22T07:04:23.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad18d8640 1 -- 172.21.15.44:0/2935090675 learned_addr learned my addr 172.21.15.44:0/2935090675 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:23.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad28da640 1 --2- 172.21.15.44:0/2935090675 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5acc15b6f0 0x7f5acc0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:23.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad20d9640 1 -- 172.21.15.44:0/2935090675 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5acc15b6f0 msgr2=0x7f5acc0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:23.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad20d9640 1 --2- 172.21.15.44:0/2935090675 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5acc15b6f0 0x7f5acc0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad20d9640 1 -- 172.21.15.44:0/2935090675 >> v1:172.21.15.44:6789/0 conn(0x7f5acc0c9df0 legacy=0x7f5acc15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:23.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad20d9640 1 -- 172.21.15.44:0/2935090675 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc070250 con 0x7f5acc1550a0 2024-04-22T07:04:23.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.208+0000 7f5ad10d7640 1 -- 172.21.15.44:0/2935090675 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993968439 0 0) 0x7f5acc0c9710 con 0x7f5acc0c9df0 2024-04-22T07:04:23.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.209+0000 7f5ad20d9640 1 --2- 172.21.15.44:0/2935090675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc1550a0 0x7f5acc0ce360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80097a0 tx=0x7f5ac802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c283f54a40ad1d67 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:23.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.209+0000 7f5ad10d7640 1 -- 172.21.15.44:0/2935090675 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8035020 con 0x7f5acc1550a0 2024-04-22T07:04:23.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.209+0000 7f5ad10d7640 1 -- 172.21.15.44:0/2935090675 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac8031830 con 0x7f5acc1550a0 2024-04-22T07:04:23.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.209+0000 7f5ad10d7640 1 -- 172.21.15.44:0/2935090675 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8031c00 con 0x7f5acc1550a0 2024-04-22T07:04:23.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.210+0000 7f5ad433d640 1 -- 172.21.15.44:0/2935090675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc1550a0 msgr2=0x7f5acc0ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:23.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.210+0000 7f5ad433d640 1 --2- 172.21.15.44:0/2935090675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc1550a0 0x7f5acc0ce360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80097a0 tx=0x7f5ac802b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.210+0000 7f5ad433d640 1 -- 172.21.15.44:0/2935090675 shutdown_connections 2024-04-22T07:04:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.210+0000 7f5ad433d640 1 --2- 172.21.15.44:0/2935090675 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5acc15b6f0 0x7f5acc0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.210+0000 7f5ad433d640 1 --2- 172.21.15.44:0/2935090675 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc1550a0 0x7f5acc0ce360 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.210+0000 7f5ad433d640 1 -- 172.21.15.44:0/2935090675 >> 172.21.15.44:0/2935090675 conn(0x7f5acc067ce0 msgr2=0x7f5acc0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.210+0000 7f5ad433d640 1 -- 172.21.15.44:0/2935090675 shutdown_connections 2024-04-22T07:04:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.210+0000 7f5ad433d640 1 -- 172.21.15.44:0/2935090675 wait complete. 2024-04-22T07:04:23.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.211+0000 7f5ad433d640 1 Processor -- start 2024-04-22T07:04:23.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.211+0000 7f5ad433d640 1 -- start start 2024-04-22T07:04:23.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad433d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0c9df0 0x7f5acc1e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:23.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad433d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5acc1550a0 0x7f5acc0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:23.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad433d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5acc15b6f0 0x7f5acc14a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:23.211 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad433d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc155480 con 0x7f5acc0c9df0 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad433d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc070810 con 0x7f5acc1550a0 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad433d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5acc155600 con 0x7f5acc15b6f0 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad20d9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0c9df0 0x7f5acc1e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad20d9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0c9df0 0x7f5acc1e4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:38180/0 (socket says 172.21.15.44:38180) 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad20d9640 1 -- 172.21.15.44:0/3385589676 learned_addr learned my addr 172.21.15.44:0/3385589676 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad28da640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5acc15b6f0 0x7f5acc14a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad20d9640 1 -- 172.21.15.44:0/3385589676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5acc15b6f0 msgr2=0x7f5acc14a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad18d8640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5acc1550a0 0x7f5acc0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad20d9640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5acc15b6f0 0x7f5acc14a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad20d9640 1 -- 172.21.15.44:0/3385589676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5acc1550a0 msgr2=0x7f5acc0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad20d9640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5acc1550a0 0x7f5acc0cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad20d9640 1 -- 172.21.15.44:0/3385589676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc1e5440 con 0x7f5acc0c9df0 2024-04-22T07:04:23.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad20d9640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0c9df0 0x7f5acc1e4f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80097a0 tx=0x7f5ac80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:23.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ac2ffd640 1 -- 172.21.15.44:0/3385589676 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8035070 con 0x7f5acc0c9df0 2024-04-22T07:04:23.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ac2ffd640 1 -- 172.21.15.44:0/3385589676 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac8004230 con 0x7f5acc0c9df0 2024-04-22T07:04:23.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ac2ffd640 1 -- 172.21.15.44:0/3385589676 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac803a6f0 con 0x7f5acc0c9df0 2024-04-22T07:04:23.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.212+0000 7f5ad433d640 1 -- 172.21.15.44:0/3385589676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc155c90 con 0x7f5acc0c9df0 2024-04-22T07:04:23.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.213+0000 7f5ad433d640 1 -- 172.21.15.44:0/3385589676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc157eb0 con 0x7f5acc0c9df0 2024-04-22T07:04:23.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.214+0000 7f5ad433d640 1 -- 172.21.15.44:0/3385589676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5acc158880 con 0x7f5acc0c9df0 2024-04-22T07:04:23.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.214+0000 7f5ac2ffd640 1 -- 172.21.15.44:0/3385589676 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ac8038020 con 0x7f5acc0c9df0 2024-04-22T07:04:23.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.214+0000 7f5ac2ffd640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5ab006c8f0 0x7f5ab006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:23.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.215+0000 7f5ac2ffd640 1 -- 172.21.15.44:0/3385589676 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5ac80ad8c0 con 0x7f5acc0c9df0 2024-04-22T07:04:23.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.215+0000 7f5ad18d8640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5ab006c8f0 0x7f5ab006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:23.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.215+0000 7f5ad18d8640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5ab006c8f0 0x7f5ab006edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5acc15f3e0 tx=0x7f5abc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:23.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.219+0000 7f5ac2ffd640 1 -- 172.21.15.44:0/3385589676 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ac8040070 con 0x7f5acc0c9df0 2024-04-22T07:04:23.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.442+0000 7f5ad433d640 1 -- 172.21.15.44:0/3385589676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5acc04fdc0 con 0x7f5acc0c9df0 2024-04-22T07:04:23.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.442+0000 7f5ac2ffd640 1 -- 172.21.15.44:0/3385589676 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f5ac8042e40 con 0x7f5acc0c9df0 2024-04-22T07:04:23.440 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:23.440 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T07:04:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 -- 172.21.15.44:0/3385589676 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5ab006c8f0 msgr2=0x7f5ab006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5ab006c8f0 0x7f5ab006edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5acc15f3e0 tx=0x7f5abc009040 comp rx=0 tx=0).stop 2024-04-22T07:04:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 -- 172.21.15.44:0/3385589676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0c9df0 msgr2=0x7f5acc1e4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0c9df0 0x7f5acc1e4f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80097a0 tx=0x7f5ac80046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 -- 172.21.15.44:0/3385589676 shutdown_connections 2024-04-22T07:04:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5ab006c8f0 0x7f5ab006edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5acc15b6f0 0x7f5acc14a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.442 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5acc1550a0 0x7f5acc0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 --2- 172.21.15.44:0/3385589676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5acc0c9df0 0x7f5acc1e4f00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:23.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 -- 172.21.15.44:0/3385589676 >> 172.21.15.44:0/3385589676 conn(0x7f5acc067ce0 msgr2=0x7f5acc06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:23.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 -- 172.21.15.44:0/3385589676 shutdown_connections 2024-04-22T07:04:23.443 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:23.444+0000 7f5ad433d640 1 -- 172.21.15.44:0/3385589676 wait complete. 2024-04-22T07:04:26.453 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T07:04:26.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.568+0000 7f8117184640 1 Processor -- start 2024-04-22T07:04:26.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.568+0000 7f8117184640 1 -- start start 2024-04-22T07:04:26.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.569+0000 7f8117184640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8110154f50 0x7f811015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.569+0000 7f8117184640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81100cb1f0 con 0x7f811015c160 2024-04-22T07:04:26.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.569+0000 7f8117184640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8110155330 con 0x7f8110072c20 2024-04-22T07:04:26.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.569+0000 7f8117184640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f81100c5420 con 0x7f8110154f50 2024-04-22T07:04:26.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.569+0000 7f8114f20640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8110154f50 0x7f811015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.569+0000 7f8115721640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f811015c160 0x7f811015c590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38028/0 (socket says 172.21.15.44:38028) 2024-04-22T07:04:26.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.569+0000 7f8115721640 1 -- 172.21.15.44:0/1204409626 learned_addr learned my addr 172.21.15.44:0/1204409626 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:26.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.570+0000 7f8114f20640 1 -- 172.21.15.44:0/1204409626 >> v1:172.21.15.70:6789/0 conn(0x7f8110072c20 legacy=0x7f81100d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:26.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.570+0000 7f8114f20640 1 -- 172.21.15.44:0/1204409626 >> v1:172.21.15.44:6789/0 conn(0x7f811015c160 legacy=0x7f811015c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.570+0000 7f8114f20640 1 -- 172.21.15.44:0/1204409626 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81100c5650 con 0x7f8110154f50 2024-04-22T07:04:26.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.570+0000 7f8114f20640 1 --2- 172.21.15.44:0/1204409626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110154f50 0x7f811015ec90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8108009d90 tx=0x7f810802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50346e9603e70cd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.570+0000 7f810f7fe640 1 -- 172.21.15.44:0/1204409626 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108039070 con 0x7f8110154f50 2024-04-22T07:04:26.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.571+0000 7f810f7fe640 1 -- 172.21.15.44:0/1204409626 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810802cd00 con 0x7f8110154f50 2024-04-22T07:04:26.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.571+0000 7f810f7fe640 1 -- 172.21.15.44:0/1204409626 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108034660 con 0x7f8110154f50 2024-04-22T07:04:26.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.571+0000 7f8117184640 1 -- 172.21.15.44:0/1204409626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110154f50 msgr2=0x7f811015ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.571+0000 7f8117184640 1 --2- 172.21.15.44:0/1204409626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110154f50 0x7f811015ec90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8108009d90 tx=0x7f810802bf50 comp rx=0 tx=0).stop 2024-04-22T07:04:26.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.571+0000 7f8117184640 1 -- 172.21.15.44:0/1204409626 shutdown_connections 2024-04-22T07:04:26.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.571+0000 7f8117184640 1 --2- 172.21.15.44:0/1204409626 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110154f50 0x7f811015ec90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.571+0000 7f8117184640 1 -- 172.21.15.44:0/1204409626 >> 172.21.15.44:0/1204409626 conn(0x7f811006caa0 msgr2=0x7f811006ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:26.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.571+0000 7f8117184640 1 -- 172.21.15.44:0/1204409626 shutdown_connections 2024-04-22T07:04:26.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.571+0000 7f8117184640 1 -- 172.21.15.44:0/1204409626 wait complete. 2024-04-22T07:04:26.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.572+0000 7f8117184640 1 Processor -- start 2024-04-22T07:04:26.570 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.572+0000 7f8117184640 1 -- start start 2024-04-22T07:04:26.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.573+0000 7f8117184640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110072c20 0x7f8110141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.573+0000 7f8117184640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8110154f50 0x7f811015e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.573+0000 7f8117184640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f811015c160 0x7f811013fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.573+0000 7f8117184640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81100c5c10 con 0x7f8110072c20 2024-04-22T07:04:26.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.573+0000 7f8117184640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f81100c6540 con 0x7f811015c160 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.573+0000 7f8117184640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81100c5ef0 con 0x7f8110154f50 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.573+0000 7f8114f20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110072c20 0x7f8110141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.573+0000 7f8114f20640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110072c20 0x7f8110141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55044/0 (socket says 172.21.15.44:55044) 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.573+0000 7f8114f20640 1 -- 172.21.15.44:0/422777103 learned_addr learned my addr 172.21.15.44:0/422777103 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f8115721640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f811015c160 0x7f811013fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f810ffff640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8110154f50 0x7f811015e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f8114f20640 1 -- 172.21.15.44:0/422777103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8110154f50 msgr2=0x7f811015e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f8114f20640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8110154f50 0x7f811015e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f8114f20640 1 -- 172.21.15.44:0/422777103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f811015c160 msgr2=0x7f811013fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f8114f20640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f811015c160 0x7f811013fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f8114f20640 1 -- 172.21.15.44:0/422777103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81101f15f0 con 0x7f8110072c20 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f8114f20640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110072c20 0x7f8110141100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8108009d90 tx=0x7f8108004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f810d7fa640 1 -- 172.21.15.44:0/422777103 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108039050 con 0x7f8110072c20 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f810d7fa640 1 -- 172.21.15.44:0/422777103 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8108045070 con 0x7f8110072c20 2024-04-22T07:04:26.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f8117184640 1 -- 172.21.15.44:0/422777103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81101f18d0 con 0x7f8110072c20 2024-04-22T07:04:26.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f810d7fa640 1 -- 172.21.15.44:0/422777103 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810803ea50 con 0x7f8110072c20 2024-04-22T07:04:26.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.574+0000 7f8117184640 1 -- 172.21.15.44:0/422777103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81101f1e90 con 0x7f8110072c20 2024-04-22T07:04:26.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.575+0000 7f8117184640 1 -- 172.21.15.44:0/422777103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81100caf30 con 0x7f8110072c20 2024-04-22T07:04:26.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.576+0000 7f810d7fa640 1 -- 172.21.15.44:0/422777103 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f810802cbc0 con 0x7f8110072c20 2024-04-22T07:04:26.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.576+0000 7f810d7fa640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80fc06c8f0 0x7f80fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:26.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.576+0000 7f810ffff640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80fc06c8f0 0x7f80fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:26.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.576+0000 7f810d7fa640 1 -- 172.21.15.44:0/422777103 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f81080ae9f0 con 0x7f8110072c20 2024-04-22T07:04:26.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.577+0000 7f810ffff640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80fc06c8f0 0x7f80fc06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f81101426c0 tx=0x7f8104009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:26.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.581+0000 7f810d7fa640 1 -- 172.21.15.44:0/422777103 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f81100caf30 con 0x7f8110072c20 2024-04-22T07:04:26.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.805+0000 7f8117184640 1 -- 172.21.15.44:0/422777103 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8110062bd0 con 0x7f8110072c20 2024-04-22T07:04:26.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.806+0000 7f810d7fa640 1 -- 172.21.15.44:0/422777103 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f8110062bd0 con 0x7f8110072c20 2024-04-22T07:04:26.804 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:26.804 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 -- 172.21.15.44:0/422777103 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80fc06c8f0 msgr2=0x7f80fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80fc06c8f0 0x7f80fc06edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f81101426c0 tx=0x7f8104009040 comp rx=0 tx=0).stop 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 -- 172.21.15.44:0/422777103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110072c20 msgr2=0x7f8110141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110072c20 0x7f8110141100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8108009d90 tx=0x7f8108004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 -- 172.21.15.44:0/422777103 shutdown_connections 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80fc06c8f0 0x7f80fc06edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8110154f50 0x7f811015e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f811015c160 0x7f811013fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 --2- 172.21.15.44:0/422777103 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110072c20 0x7f8110141100 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 -- 172.21.15.44:0/422777103 >> 172.21.15.44:0/422777103 conn(0x7f811006caa0 msgr2=0x7f8110152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:26.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 -- 172.21.15.44:0/422777103 shutdown_connections 2024-04-22T07:04:26.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:26.808+0000 7f8117184640 1 -- 172.21.15.44:0/422777103 wait complete. 2024-04-22T07:04:28.140 INFO:tasks.ceph.mds.f.smithi070.stderr:2024-04-22T07:04:28.141+0000 7f78ff051640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-22T07:04:29.821 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T07:04:29.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.934+0000 7f33c426a640 1 Processor -- start 2024-04-22T07:04:29.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.934+0000 7f33c426a640 1 -- start start 2024-04-22T07:04:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.935+0000 7f33c426a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f33bc1561d0 0x7f33bc0c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.935+0000 7f33c426a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0b3020 0x7f33bc0ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.935+0000 7f33c426a640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33bc0c91a0 con 0x7f33bc0ce340 2024-04-22T07:04:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.935+0000 7f33c426a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f33bc0701b0 con 0x7f33bc1561d0 2024-04-22T07:04:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.935+0000 7f33c426a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc070490 con 0x7f33bc0b3020 2024-04-22T07:04:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.935+0000 7f33c2807640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f33bc0ce340 0x7f33bc0cae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:38036/0 (socket says 172.21.15.44:38036) 2024-04-22T07:04:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.935+0000 7f33c2807640 1 -- 172.21.15.44:0/361942201 learned_addr learned my addr 172.21.15.44:0/361942201 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.935+0000 7f33c2006640 1 --2- 172.21.15.44:0/361942201 >> v2:172.21.15.70:3300/0 conn(0x7f33bc1561d0 0x7f33bc0c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.935+0000 7f33c1805640 1 --2- 172.21.15.44:0/361942201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0b3020 0x7f33bc0ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.937 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.936+0000 7f33c1004640 1 -- 172.21.15.44:0/361942201 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252670523 0 0) 0x7f33bc0c91a0 con 0x7f33bc0ce340 2024-04-22T07:04:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.936+0000 7f33c1004640 1 -- 172.21.15.44:0/361942201 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33a8003520 con 0x7f33bc0ce340 2024-04-22T07:04:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.936+0000 7f33c1805640 1 -- 172.21.15.44:0/361942201 >> v2:172.21.15.70:3300/0 conn(0x7f33bc1561d0 msgr2=0x7f33bc0c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.936+0000 7f33c1805640 1 --2- 172.21.15.44:0/361942201 >> v2:172.21.15.70:3300/0 conn(0x7f33bc1561d0 0x7f33bc0c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.936+0000 7f33c1805640 1 -- 172.21.15.44:0/361942201 >> v1:172.21.15.44:6789/0 conn(0x7f33bc0ce340 legacy=0x7f33bc0cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.936+0000 7f33c1805640 1 -- 172.21.15.44:0/361942201 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc070770 con 0x7f33bc0b3020 2024-04-22T07:04:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.936+0000 7f33c2006640 1 --2- 172.21.15.44:0/361942201 >> v2:172.21.15.70:3300/0 conn(0x7f33bc1561d0 0x7f33bc0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c1805640 1 --2- 172.21.15.44:0/361942201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0b3020 0x7f33bc0ca5c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0097a0 tx=0x7f33ac02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e44b4954844d2c3c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:29.938 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c1004640 1 -- 172.21.15.44:0/361942201 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac035020 con 0x7f33bc0b3020 2024-04-22T07:04:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c1004640 1 -- 172.21.15.44:0/361942201 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac031830 con 0x7f33bc0b3020 2024-04-22T07:04:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c1004640 1 -- 172.21.15.44:0/361942201 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac031c00 con 0x7f33bc0b3020 2024-04-22T07:04:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c426a640 1 -- 172.21.15.44:0/361942201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0b3020 msgr2=0x7f33bc0ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c426a640 1 --2- 172.21.15.44:0/361942201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0b3020 0x7f33bc0ca5c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0097a0 tx=0x7f33ac02b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c426a640 1 -- 172.21.15.44:0/361942201 shutdown_connections 2024-04-22T07:04:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c426a640 1 --2- 172.21.15.44:0/361942201 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0b3020 0x7f33bc0ca5c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c426a640 1 --2- 172.21.15.44:0/361942201 >> v2:172.21.15.70:3300/0 conn(0x7f33bc1561d0 0x7f33bc0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c426a640 1 -- 172.21.15.44:0/361942201 >> 172.21.15.44:0/361942201 conn(0x7f33bc067ce0 msgr2=0x7f33bc0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:29.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c426a640 1 -- 172.21.15.44:0/361942201 shutdown_connections 2024-04-22T07:04:29.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.937+0000 7f33c426a640 1 -- 172.21.15.44:0/361942201 wait complete. 2024-04-22T07:04:29.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.939+0000 7f33c426a640 1 Processor -- start 2024-04-22T07:04:29.940 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.939+0000 7f33c426a640 1 -- start start 2024-04-22T07:04:29.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.939+0000 7f33c426a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc0b3020 0x7f33bc1f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.943 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.939+0000 7f33c426a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0ce340 0x7f33bc15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.939+0000 7f33c426a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc1561d0 0x7f33bc152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.939+0000 7f33c426a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1565b0 con 0x7f33bc0b3020 2024-04-22T07:04:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.939+0000 7f33c426a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc0cab00 con 0x7f33bc1561d0 2024-04-22T07:04:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.939+0000 7f33c426a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc0c8dd0 con 0x7f33bc0ce340 2024-04-22T07:04:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c2006640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc0b3020 0x7f33bc1f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.944 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c1805640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0ce340 0x7f33bc15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c2807640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc1561d0 0x7f33bc152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c2006640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc0b3020 0x7f33bc1f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:55046/0 (socket says 172.21.15.44:55046) 2024-04-22T07:04:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c1805640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0ce340 0x7f33bc15a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:49012/0 (socket says 172.21.15.44:49012) 2024-04-22T07:04:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c2006640 1 -- 172.21.15.44:0/1244929114 learned_addr learned my addr 172.21.15.44:0/1244929114 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c1805640 1 -- 172.21.15.44:0/1244929114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc1561d0 msgr2=0x7f33bc152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c1805640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc1561d0 0x7f33bc152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c1805640 1 -- 172.21.15.44:0/1244929114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc0b3020 msgr2=0x7f33bc1f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c1805640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc0b3020 0x7f33bc1f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c1805640 1 -- 172.21.15.44:0/1244929114 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc15b7f0 con 0x7f33bc0ce340 2024-04-22T07:04:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c2006640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc0b3020 0x7f33bc1f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c2807640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc1561d0 0x7f33bc152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c1805640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0ce340 0x7f33bc15a870 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f33ac033000 tx=0x7f33ac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33b2ffd640 1 -- 172.21.15.44:0/1244929114 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac035040 con 0x7f33bc0ce340 2024-04-22T07:04:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33b2ffd640 1 -- 172.21.15.44:0/1244929114 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac03b840 con 0x7f33bc0ce340 2024-04-22T07:04:29.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c426a640 1 -- 172.21.15.44:0/1244929114 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc159d90 con 0x7f33bc0ce340 2024-04-22T07:04:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33b2ffd640 1 -- 172.21.15.44:0/1244929114 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac03bbf0 con 0x7f33bc0ce340 2024-04-22T07:04:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.940+0000 7f33c426a640 1 -- 172.21.15.44:0/1244929114 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc155850 con 0x7f33bc0ce340 2024-04-22T07:04:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.941+0000 7f33c426a640 1 -- 172.21.15.44:0/1244929114 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33bc0c9dd0 con 0x7f33bc0ce340 2024-04-22T07:04:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.942+0000 7f33b2ffd640 1 -- 172.21.15.44:0/1244929114 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33ac038020 con 0x7f33bc0ce340 2024-04-22T07:04:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.942+0000 7f33b2ffd640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c8f0 0x7f33a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:29.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.942+0000 7f33c2006640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c8f0 0x7f33a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.942+0000 7f33b2ffd640 1 -- 172.21.15.44:0/1244929114 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f33ac0ada40 con 0x7f33bc0ce340 2024-04-22T07:04:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.943+0000 7f33c2006640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c8f0 0x7f33a006edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33b40095e0 tx=0x7f33b4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:29.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:29.947+0000 7f33b2ffd640 1 -- 172.21.15.44:0/1244929114 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f33ac07b820 con 0x7f33bc0ce340 2024-04-22T07:04:30.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.170+0000 7f33c426a640 1 -- 172.21.15.44:0/1244929114 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f33bc04fdc0 con 0x7f33bc0ce340 2024-04-22T07:04:30.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.170+0000 7f33b2ffd640 1 -- 172.21.15.44:0/1244929114 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f33ac0ade20 con 0x7f33bc0ce340 2024-04-22T07:04:30.168 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:30.168 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T07:04:30.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 -- 172.21.15.44:0/1244929114 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c8f0 msgr2=0x7f33a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:30.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c8f0 0x7f33a006edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f33b40095e0 tx=0x7f33b4009190 comp rx=0 tx=0).stop 2024-04-22T07:04:30.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 -- 172.21.15.44:0/1244929114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0ce340 msgr2=0x7f33bc15a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:30.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0ce340 0x7f33bc15a870 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f33ac033000 tx=0x7f33ac0046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:30.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 -- 172.21.15.44:0/1244929114 shutdown_connections 2024-04-22T07:04:30.170 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f33a006c8f0 0x7f33a006edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:30.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f33bc0ce340 0x7f33bc15a870 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:30.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc1561d0 0x7f33bc152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:30.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 --2- 172.21.15.44:0/1244929114 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f33bc0b3020 0x7f33bc1f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:30.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 -- 172.21.15.44:0/1244929114 >> 172.21.15.44:0/1244929114 conn(0x7f33bc067ce0 msgr2=0x7f33bc071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:30.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 -- 172.21.15.44:0/1244929114 shutdown_connections 2024-04-22T07:04:30.171 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:30.172+0000 7f33c426a640 1 -- 172.21.15.44:0/1244929114 wait complete. 2024-04-22T07:04:33.181 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T07:04:33.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.295+0000 7fd750874640 1 Processor -- start 2024-04-22T07:04:33.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.296+0000 7fd750874640 1 -- start start 2024-04-22T07:04:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd750874640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd74c0cc880 0x7fd74c0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd750874640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0c5a00 0x7fd74c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd750874640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd74c15f1b0 con 0x7fd74c154f50 2024-04-22T07:04:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd750874640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd74c0cb7c0 con 0x7fd74c0cc880 2024-04-22T07:04:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd750874640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd74c15e3f0 con 0x7fd74c0c5a00 2024-04-22T07:04:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd74ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0c5a00 0x7fd74c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd749d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd74c0cc880 0x7fd74c0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd74ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0c5a00 0x7fd74c0ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:49026/0 (socket says 172.21.15.44:49026) 2024-04-22T07:04:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd74ad9d640 1 -- 172.21.15.44:0/2675207163 learned_addr learned my addr 172.21.15.44:0/2675207163 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:33.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd74a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd74c154f50 0x7fd74c15b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:40462/0 (socket says 172.21.15.44:40462) 2024-04-22T07:04:33.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd74ad9d640 1 -- 172.21.15.44:0/2675207163 >> v2:172.21.15.70:3300/0 conn(0x7fd74c0cc880 msgr2=0x7fd74c0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:33.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd74ad9d640 1 --2- 172.21.15.44:0/2675207163 >> v2:172.21.15.70:3300/0 conn(0x7fd74c0cc880 0x7fd74c0c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd74ad9d640 1 -- 172.21.15.44:0/2675207163 >> v1:172.21.15.70:6789/0 conn(0x7fd74c154f50 legacy=0x7fd74c15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:33.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd74ad9d640 1 -- 172.21.15.44:0/2675207163 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74c15e6d0 con 0x7fd74c0c5a00 2024-04-22T07:04:33.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd74959a640 1 -- 172.21.15.44:0/2675207163 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935671438 0 0) 0x7fd74c15f1b0 con 0x7fd74c154f50 2024-04-22T07:04:33.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.297+0000 7fd749d9b640 1 --2- 172.21.15.44:0/2675207163 >> v2:172.21.15.70:3300/0 conn(0x7fd74c0cc880 0x7fd74c0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:33.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.298+0000 7fd74ad9d640 1 --2- 172.21.15.44:0/2675207163 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0c5a00 0x7fd74c0ccf30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009e50 tx=0x7fd73c02bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad105352615e0c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:33.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.298+0000 7fd74959a640 1 -- 172.21.15.44:0/2675207163 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c02cb90 con 0x7fd74c0c5a00 2024-04-22T07:04:33.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.298+0000 7fd74959a640 1 -- 172.21.15.44:0/2675207163 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c02cd30 con 0x7fd74c0c5a00 2024-04-22T07:04:33.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.298+0000 7fd74959a640 1 -- 172.21.15.44:0/2675207163 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c034610 con 0x7fd74c0c5a00 2024-04-22T07:04:33.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.299+0000 7fd750874640 1 -- 172.21.15.44:0/2675207163 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0c5a00 msgr2=0x7fd74c0ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:33.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.299+0000 7fd750874640 1 --2- 172.21.15.44:0/2675207163 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0c5a00 0x7fd74c0ccf30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd73c009e50 tx=0x7fd73c02bff0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.299+0000 7fd750874640 1 -- 172.21.15.44:0/2675207163 shutdown_connections 2024-04-22T07:04:33.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.299+0000 7fd750874640 1 --2- 172.21.15.44:0/2675207163 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0c5a00 0x7fd74c0ccf30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.299+0000 7fd750874640 1 --2- 172.21.15.44:0/2675207163 >> v2:172.21.15.70:3300/0 conn(0x7fd74c0cc880 0x7fd74c0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.299+0000 7fd750874640 1 -- 172.21.15.44:0/2675207163 >> 172.21.15.44:0/2675207163 conn(0x7fd74c067d40 msgr2=0x7fd74c06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:33.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.299+0000 7fd750874640 1 -- 172.21.15.44:0/2675207163 shutdown_connections 2024-04-22T07:04:33.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.299+0000 7fd750874640 1 -- 172.21.15.44:0/2675207163 wait complete. 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.300+0000 7fd750874640 1 Processor -- start 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.300+0000 7fd750874640 1 -- start start 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd750874640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd74c0c5a00 0x7fd74c153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd750874640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0cc880 0x7fd74c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd750874640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd74c154f50 0x7fd74c0cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd750874640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd74c155330 con 0x7fd74c154f50 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd750874640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd74c15ec90 con 0x7fd74c0c5a00 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd750874640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd74c0cb430 con 0x7fd74c0cc880 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd74a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd74c0c5a00 0x7fd74c153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd749d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0cc880 0x7fd74c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:33.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd74ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd74c154f50 0x7fd74c0cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd749d9b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0cc880 0x7fd74c1592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:49032/0 (socket says 172.21.15.44:49032) 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd74a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd74c0c5a00 0x7fd74c153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:54686/0 (socket says 172.21.15.44:54686) 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd749d9b640 1 -- 172.21.15.44:0/2250216082 learned_addr learned my addr 172.21.15.44:0/2250216082 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd749d9b640 1 -- 172.21.15.44:0/2250216082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd74c0c5a00 msgr2=0x7fd74c153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd749d9b640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd74c0c5a00 0x7fd74c153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd749d9b640 1 -- 172.21.15.44:0/2250216082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd74c154f50 msgr2=0x7fd74c0cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd749d9b640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd74c154f50 0x7fd74c0cfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd749d9b640 1 -- 172.21.15.44:0/2250216082 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74c15a590 con 0x7fd74c0cc880 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.301+0000 7fd74a59c640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd74c0c5a00 0x7fd74c153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.302+0000 7fd749d9b640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0cc880 0x7fd74c1592d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd73400ec10 tx=0x7fd73400c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.302+0000 7fd742ffd640 1 -- 172.21.15.44:0/2250216082 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734017070 con 0x7fd74c0cc880 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.302+0000 7fd742ffd640 1 -- 172.21.15.44:0/2250216082 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73400cc30 con 0x7fd74c0cc880 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.302+0000 7fd750874640 1 -- 172.21.15.44:0/2250216082 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74c15a870 con 0x7fd74c0cc880 2024-04-22T07:04:33.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.302+0000 7fd742ffd640 1 -- 172.21.15.44:0/2250216082 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734013470 con 0x7fd74c0cc880 2024-04-22T07:04:33.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.302+0000 7fd750874640 1 -- 172.21.15.44:0/2250216082 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74c0cf0c0 con 0x7fd74c0cc880 2024-04-22T07:04:33.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.303+0000 7fd750874640 1 -- 172.21.15.44:0/2250216082 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd74c04fdc0 con 0x7fd74c0cc880 2024-04-22T07:04:33.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.303+0000 7fd742ffd640 1 -- 172.21.15.44:0/2250216082 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd73401a070 con 0x7fd74c0cc880 2024-04-22T07:04:33.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.304+0000 7fd742ffd640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd72c06c920 0x7fd72c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:33.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.304+0000 7fd742ffd640 1 -- 172.21.15.44:0/2250216082 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd73408ba20 con 0x7fd74c0cc880 2024-04-22T07:04:33.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.304+0000 7fd74a59c640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd72c06c920 0x7fd72c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:33.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.304+0000 7fd74a59c640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd72c06c920 0x7fd72c06ede0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd744007fd0 tx=0x7fd744007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:33.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.308+0000 7fd742ffd640 1 -- 172.21.15.44:0/2250216082 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd734059800 con 0x7fd74c0cc880 2024-04-22T07:04:33.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.532+0000 7fd750874640 1 -- 172.21.15.44:0/2250216082 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd74c0cbf70 con 0x7fd74c0cc880 2024-04-22T07:04:33.530 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.533+0000 7fd742ffd640 1 -- 172.21.15.44:0/2250216082 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7fd73405db00 con 0x7fd74c0cc880 2024-04-22T07:04:33.531 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:33.531 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T07:04:33.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 -- 172.21.15.44:0/2250216082 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd72c06c920 msgr2=0x7fd72c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:33.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd72c06c920 0x7fd72c06ede0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd744007fd0 tx=0x7fd744007de0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 -- 172.21.15.44:0/2250216082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0cc880 msgr2=0x7fd74c1592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:33.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0cc880 0x7fd74c1592d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd73400ec10 tx=0x7fd73400c520 comp rx=0 tx=0).stop 2024-04-22T07:04:33.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 -- 172.21.15.44:0/2250216082 shutdown_connections 2024-04-22T07:04:33.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd72c06c920 0x7fd72c06ede0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd74c0cc880 0x7fd74c1592d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd74c154f50 0x7fd74c0cfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 --2- 172.21.15.44:0/2250216082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd74c0c5a00 0x7fd74c153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:33.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 -- 172.21.15.44:0/2250216082 >> 172.21.15.44:0/2250216082 conn(0x7fd74c067d40 msgr2=0x7fd74c0b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:33.533 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 -- 172.21.15.44:0/2250216082 shutdown_connections 2024-04-22T07:04:33.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:33.535+0000 7fd750874640 1 -- 172.21.15.44:0/2250216082 wait complete. 2024-04-22T07:04:36.544 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.658+0000 7fea4c38f640 1 Processor -- start 2024-04-22T07:04:36.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.658+0000 7fea4c38f640 1 -- start start 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.659+0000 7fea4c38f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea44155350 0x7fea440b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.659+0000 7fea4c38f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44072d10 0x7fea440b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.659+0000 7fea4c38f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea4415b180 con 0x7fea44154f70 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.659+0000 7fea4c38f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fea4415b460 con 0x7fea44155350 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.659+0000 7fea4c38f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fea4415b740 con 0x7fea44072d10 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.660+0000 7fea4a92c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44072d10 0x7fea440b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.660+0000 7fea4a12b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fea44154f70 0x7fea440d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:45240/0 (socket says 172.21.15.44:45240) 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.660+0000 7fea4a12b640 1 -- 172.21.15.44:0/1872056325 learned_addr learned my addr 172.21.15.44:0/1872056325 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.660+0000 7fea4992a640 1 --2- 172.21.15.44:0/1872056325 >> v2:172.21.15.70:3300/0 conn(0x7fea44155350 0x7fea440b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.660+0000 7fea4a92c640 1 -- 172.21.15.44:0/1872056325 >> v2:172.21.15.70:3300/0 conn(0x7fea44155350 msgr2=0x7fea440b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:36.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.660+0000 7fea4a92c640 1 --2- 172.21.15.44:0/1872056325 >> v2:172.21.15.70:3300/0 conn(0x7fea44155350 0x7fea440b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.660+0000 7fea49129640 1 -- 172.21.15.44:0/1872056325 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197853213 0 0) 0x7fea4415b180 con 0x7fea44154f70 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.660+0000 7fea4a92c640 1 -- 172.21.15.44:0/1872056325 >> v1:172.21.15.70:6789/0 conn(0x7fea44154f70 legacy=0x7fea440d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.660+0000 7fea4a92c640 1 -- 172.21.15.44:0/1872056325 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4415ba20 con 0x7fea44072d10 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.661+0000 7fea4a92c640 1 --2- 172.21.15.44:0/1872056325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44072d10 0x7fea440b48a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009d90 tx=0x7fea3c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6c5bf99e638ef5e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.661+0000 7fea49129640 1 -- 172.21.15.44:0/1872056325 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c039070 con 0x7fea44072d10 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.661+0000 7fea49129640 1 -- 172.21.15.44:0/1872056325 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3c02cc50 con 0x7fea44072d10 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.661+0000 7fea49129640 1 -- 172.21.15.44:0/1872056325 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c034440 con 0x7fea44072d10 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.662+0000 7fea4c38f640 1 -- 172.21.15.44:0/1872056325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44072d10 msgr2=0x7fea440b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.662+0000 7fea4c38f640 1 --2- 172.21.15.44:0/1872056325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44072d10 0x7fea440b48a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009d90 tx=0x7fea3c02bea0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.662+0000 7fea4c38f640 1 -- 172.21.15.44:0/1872056325 shutdown_connections 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.662+0000 7fea4c38f640 1 --2- 172.21.15.44:0/1872056325 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44072d10 0x7fea440b48a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.662+0000 7fea4c38f640 1 --2- 172.21.15.44:0/1872056325 >> v2:172.21.15.70:3300/0 conn(0x7fea44155350 0x7fea440b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.662+0000 7fea4c38f640 1 -- 172.21.15.44:0/1872056325 >> 172.21.15.44:0/1872056325 conn(0x7fea440735c0 msgr2=0x7fea440739d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.662+0000 7fea4c38f640 1 -- 172.21.15.44:0/1872056325 shutdown_connections 2024-04-22T07:04:36.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.662+0000 7fea4c38f640 1 -- 172.21.15.44:0/1872056325 wait complete. 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.663+0000 7fea4c38f640 1 Processor -- start 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.663+0000 7fea4c38f640 1 -- start start 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4c38f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea44072d10 0x7fea441f1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4c38f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea44154f70 0x7fea440cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4c38f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44155c70 0x7fea44149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4c38f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fea4415add0 con 0x7fea44154f70 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4c38f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fea440d1010 con 0x7fea44072d10 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4c38f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fea441f1560 con 0x7fea44155c70 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4992a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea44154f70 0x7fea440cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4a12b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea44072d10 0x7fea441f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4a12b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea44072d10 0x7fea441f1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:54624/0 (socket says 172.21.15.44:54624) 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4a12b640 1 -- 172.21.15.44:0/2759076275 learned_addr learned my addr 172.21.15.44:0/2759076275 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4a92c640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44155c70 0x7fea44149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4a12b640 1 -- 172.21.15.44:0/2759076275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44155c70 msgr2=0x7fea44149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4a12b640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44155c70 0x7fea44149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4a12b640 1 -- 172.21.15.44:0/2759076275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea44154f70 msgr2=0x7fea440cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4a12b640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea44154f70 0x7fea440cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4a12b640 1 -- 172.21.15.44:0/2759076275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea44150d60 con 0x7fea44072d10 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4a12b640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea44072d10 0x7fea441f1020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c490 tx=0x7fea4000c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea3affd640 1 -- 172.21.15.44:0/2759076275 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40018070 con 0x7fea44072d10 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea3affd640 1 -- 172.21.15.44:0/2759076275 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea40007500 con 0x7fea44072d10 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea4c38f640 1 -- 172.21.15.44:0/2759076275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea44156f20 con 0x7fea44072d10 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.664+0000 7fea3affd640 1 -- 172.21.15.44:0/2759076275 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40004a20 con 0x7fea44072d10 2024-04-22T07:04:36.664 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.665+0000 7fea4c38f640 1 -- 172.21.15.44:0/2759076275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea441574e0 con 0x7fea44072d10 2024-04-22T07:04:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.665+0000 7fea4c38f640 1 -- 172.21.15.44:0/2759076275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea440cf3b0 con 0x7fea44072d10 2024-04-22T07:04:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.666+0000 7fea3affd640 1 -- 172.21.15.44:0/2759076275 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea4001b070 con 0x7fea44072d10 2024-04-22T07:04:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.666+0000 7fea3affd640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fea2806c920 0x7fea2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.666+0000 7fea3affd640 1 -- 172.21.15.44:0/2759076275 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fea4008ae90 con 0x7fea44072d10 2024-04-22T07:04:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.666+0000 7fea4992a640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fea2806c920 0x7fea2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:36.665 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.667+0000 7fea4992a640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fea2806c920 0x7fea2806ede0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fea44152080 tx=0x7fea34009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:36.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.671+0000 7fea3affd640 1 -- 172.21.15.44:0/2759076275 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fea440cf3b0 con 0x7fea44072d10 2024-04-22T07:04:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.896+0000 7fea4c38f640 1 -- 172.21.15.44:0/2759076275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fea440c6f80 con 0x7fea44072d10 2024-04-22T07:04:36.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.897+0000 7fea3affd640 1 -- 172.21.15.44:0/2759076275 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7fea440c6f80 con 0x7fea44072d10 2024-04-22T07:04:36.895 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:36.895 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-04-22T07:04:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 -- 172.21.15.44:0/2759076275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fea2806c920 msgr2=0x7fea2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fea2806c920 0x7fea2806ede0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fea44152080 tx=0x7fea34009190 comp rx=0 tx=0).stop 2024-04-22T07:04:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 -- 172.21.15.44:0/2759076275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea44072d10 msgr2=0x7fea441f1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea44072d10 0x7fea441f1020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea4000c490 tx=0x7fea4000c950 comp rx=0 tx=0).stop 2024-04-22T07:04:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 -- 172.21.15.44:0/2759076275 shutdown_connections 2024-04-22T07:04:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fea2806c920 0x7fea2806ede0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fea44155c70 0x7fea44149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fea44154f70 0x7fea440cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 --2- 172.21.15.44:0/2759076275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea44072d10 0x7fea441f1020 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:36.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 -- 172.21.15.44:0/2759076275 >> 172.21.15.44:0/2759076275 conn(0x7fea440735c0 msgr2=0x7fea440cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:36.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 -- 172.21.15.44:0/2759076275 shutdown_connections 2024-04-22T07:04:36.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:36.899+0000 7fea4c38f640 1 -- 172.21.15.44:0/2759076275 wait complete. 2024-04-22T07:04:39.908 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T07:04:40.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.024+0000 7fe2e14a8640 1 Processor -- start 2024-04-22T07:04:40.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.024+0000 7fe2e14a8640 1 -- start start 2024-04-22T07:04:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2e14a8640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe2dc154f50 0x7fe2dc15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2e14a8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2dc0cc880 0x7fe2dc0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2e14a8640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2dc15bc20 con 0x7fe2dc0c5a00 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2e14a8640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fe2dc15f1b0 con 0x7fe2dc154f50 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2e14a8640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe2dc0cb7c0 con 0x7fe2dc0cc880 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2daffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe2dc154f50 0x7fe2dc15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2daffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe2dc154f50 0x7fe2dc15b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41022/0 (socket says 172.21.15.44:41022) 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2daffd640 1 -- 172.21.15.44:0/2513387146 learned_addr learned my addr 172.21.15.44:0/2513387146 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2db7fe640 1 --1- 172.21.15.44:0/2513387146 >> v1:172.21.15.44:6789/0 conn(0x7fe2dc0c5a00 0x7fe2dc0ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55626/0 (socket says 172.21.15.44:55626) 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2da7fc640 1 --2- 172.21.15.44:0/2513387146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2dc0cc880 0x7fe2dc0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2daffd640 1 -- 172.21.15.44:0/2513387146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2dc0cc880 msgr2=0x7fe2dc0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2daffd640 1 --2- 172.21.15.44:0/2513387146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2dc0cc880 0x7fe2dc0c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.025+0000 7fe2daffd640 1 -- 172.21.15.44:0/2513387146 >> v1:172.21.15.44:6789/0 conn(0x7fe2dc0c5a00 legacy=0x7fe2dc0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.026+0000 7fe2daffd640 1 -- 172.21.15.44:0/2513387146 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc15e3f0 con 0x7fe2dc154f50 2024-04-22T07:04:40.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.026+0000 7fe2d9ffb640 1 -- 172.21.15.44:0/2513387146 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2970461977 0 0) 0x7fe2cc001490 con 0x7fe2dc0c5a00 2024-04-22T07:04:40.025 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.026+0000 7fe2daffd640 1 --2- 172.21.15.44:0/2513387146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 0x7fe2dc15b6e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40098a0 tx=0x7fe2d402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4170b9719cf590 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.026+0000 7fe2d9ffb640 1 -- 172.21.15.44:0/2513387146 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d402f830 con 0x7fe2dc154f50 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.026+0000 7fe2d9ffb640 1 -- 172.21.15.44:0/2513387146 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d402f9d0 con 0x7fe2dc154f50 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.026+0000 7fe2d9ffb640 1 -- 172.21.15.44:0/2513387146 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d402fda0 con 0x7fe2dc154f50 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.027+0000 7fe2e14a8640 1 -- 172.21.15.44:0/2513387146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 msgr2=0x7fe2dc15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.027+0000 7fe2e14a8640 1 --2- 172.21.15.44:0/2513387146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 0x7fe2dc15b6e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40098a0 tx=0x7fe2d402ba10 comp rx=0 tx=0).stop 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.027+0000 7fe2e14a8640 1 -- 172.21.15.44:0/2513387146 shutdown_connections 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.027+0000 7fe2e14a8640 1 --2- 172.21.15.44:0/2513387146 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2dc0cc880 0x7fe2dc0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.027+0000 7fe2e14a8640 1 --2- 172.21.15.44:0/2513387146 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 0x7fe2dc15b6e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.027+0000 7fe2e14a8640 1 -- 172.21.15.44:0/2513387146 >> 172.21.15.44:0/2513387146 conn(0x7fe2dc067d40 msgr2=0x7fe2dc06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.027+0000 7fe2e14a8640 1 -- 172.21.15.44:0/2513387146 shutdown_connections 2024-04-22T07:04:40.026 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.027+0000 7fe2e14a8640 1 -- 172.21.15.44:0/2513387146 wait complete. 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.028+0000 7fe2e14a8640 1 Processor -- start 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.028+0000 7fe2e14a8640 1 -- start start 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2e14a8640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2dc0c5a00 0x7fe2dc15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2e14a8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2dc0cc880 0x7fe2dc159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2e14a8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 0x7fe2dc1e5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2e14a8640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc15ec90 con 0x7fe2dc154f50 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2e14a8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc15e9b0 con 0x7fe2dc0cc880 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2e14a8640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe2dc155330 con 0x7fe2dc0c5a00 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2db7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 0x7fe2dc1e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2db7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 0x7fe2dc1e5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41036/0 (socket says 172.21.15.44:41036) 2024-04-22T07:04:40.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2db7fe640 1 -- 172.21.15.44:0/4171543843 learned_addr learned my addr 172.21.15.44:0/4171543843 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2da7fc640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2dc0cc880 0x7fe2dc159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.029+0000 7fe2daffd640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2dc0c5a00 0x7fe2dc15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2db7fe640 1 -- 172.21.15.44:0/4171543843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2dc0c5a00 msgr2=0x7fe2dc15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2db7fe640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2dc0c5a00 0x7fe2dc15ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2db7fe640 1 -- 172.21.15.44:0/4171543843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2dc0cc880 msgr2=0x7fe2dc159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2db7fe640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2dc0cc880 0x7fe2dc159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2db7fe640 1 -- 172.21.15.44:0/4171543843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc15b4a0 con 0x7fe2dc154f50 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2db7fe640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 0x7fe2dc1e5090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00c820 tx=0x7fe2cc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2d37fe640 1 -- 172.21.15.44:0/4171543843 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc018070 con 0x7fe2dc154f50 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2d37fe640 1 -- 172.21.15.44:0/4171543843 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2cc00a5e0 con 0x7fe2dc154f50 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2e14a8640 1 -- 172.21.15.44:0/4171543843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2dc15b780 con 0x7fe2dc154f50 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2d37fe640 1 -- 172.21.15.44:0/4171543843 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc00a9d0 con 0x7fe2dc154f50 2024-04-22T07:04:40.030 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.030+0000 7fe2e14a8640 1 -- 172.21.15.44:0/4171543843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2dc158d10 con 0x7fe2dc154f50 2024-04-22T07:04:40.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.031+0000 7fe2e14a8640 1 -- 172.21.15.44:0/4171543843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2dc158820 con 0x7fe2dc154f50 2024-04-22T07:04:40.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.032+0000 7fe2d37fe640 1 -- 172.21.15.44:0/4171543843 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2cc01b070 con 0x7fe2dc154f50 2024-04-22T07:04:40.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.032+0000 7fe2d37fe640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.032+0000 7fe2d37fe640 1 -- 172.21.15.44:0/4171543843 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe2cc08b0d0 con 0x7fe2dc154f50 2024-04-22T07:04:40.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.032+0000 7fe2daffd640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.033+0000 7fe2daffd640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4036000 tx=0x7fe2d4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.037+0000 7fe2d37fe640 1 -- 172.21.15.44:0/4171543843 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2dc15bc20 con 0x7fe2dc154f50 2024-04-22T07:04:40.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.262+0000 7fe2e14a8640 1 -- 172.21.15.44:0/4171543843 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe2dc0c8a00 con 0x7fe2dc154f50 2024-04-22T07:04:40.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.263+0000 7fe2d37fe640 1 -- 172.21.15.44:0/4171543843 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe2cc058eb0 con 0x7fe2dc154f50 2024-04-22T07:04:40.260 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:40.261 INFO:teuthology.orchestra.run.smithi044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 -- 172.21.15.44:0/4171543843 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe2bc06c8f0 msgr2=0x7fe2bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4036000 tx=0x7fe2d4008030 comp rx=0 tx=0).stop 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 -- 172.21.15.44:0/4171543843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 msgr2=0x7fe2dc1e5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 0x7fe2dc1e5090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00c820 tx=0x7fe2cc00cce0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 -- 172.21.15.44:0/4171543843 shutdown_connections 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe2bc06c8f0 0x7fe2bc06edb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe2dc154f50 0x7fe2dc1e5090 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2dc0cc880 0x7fe2dc159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 --2- 172.21.15.44:0/4171543843 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe2dc0c5a00 0x7fe2dc15ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 -- 172.21.15.44:0/4171543843 >> 172.21.15.44:0/4171543843 conn(0x7fe2dc067d40 msgr2=0x7fe2dc0b4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 -- 172.21.15.44:0/4171543843 shutdown_connections 2024-04-22T07:04:40.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.265+0000 7fe2e14a8640 1 -- 172.21.15.44:0/4171543843 wait complete. 2024-04-22T07:04:40.273 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-22T07:04:40.274 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:40.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.376+0000 7fd8a73e6640 1 Processor -- start 2024-04-22T07:04:40.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.376+0000 7fd8a73e6640 1 -- start start 2024-04-22T07:04:40.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.377+0000 7fd8a73e6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8a0154f50 0x7fd8a015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.377+0000 7fd8a73e6640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd8a015c160 0x7fd8a015c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.377+0000 7fd8a73e6640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a00cb1f0 con 0x7fd8a0072c20 2024-04-22T07:04:40.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.377+0000 7fd8a73e6640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd8a00c5420 con 0x7fd8a015c160 2024-04-22T07:04:40.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.377+0000 7fd8a73e6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fd8a00c6540 con 0x7fd8a0154f50 2024-04-22T07:04:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.377+0000 7fd8a5983640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd8a015c160 0x7fd8a015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.377+0000 7fd8a4981640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fd8a0072c20 0x7fd8a00d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55638/0 (socket says 172.21.15.44:55638) 2024-04-22T07:04:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.377+0000 7fd8a4981640 1 -- 172.21.15.44:0/3230004973 learned_addr learned my addr 172.21.15.44:0/3230004973 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.377+0000 7fd8a5983640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd8a015c160 0x7fd8a015c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41052/0 (socket says 172.21.15.44:41052) 2024-04-22T07:04:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.377+0000 7fd8a5182640 1 --2- 172.21.15.44:0/3230004973 >> v2:172.21.15.70:3300/0 conn(0x7fd8a0154f50 0x7fd8a015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.378+0000 7fd8a5983640 1 -- 172.21.15.44:0/3230004973 >> v2:172.21.15.70:3300/0 conn(0x7fd8a0154f50 msgr2=0x7fd8a015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.378+0000 7fd8a5983640 1 --2- 172.21.15.44:0/3230004973 >> v2:172.21.15.70:3300/0 conn(0x7fd8a0154f50 0x7fd8a015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.378+0000 7fd8a5983640 1 -- 172.21.15.44:0/3230004973 >> v1:172.21.15.44:6789/0 conn(0x7fd8a0072c20 legacy=0x7fd8a00d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.378+0000 7fd8a5983640 1 -- 172.21.15.44:0/3230004973 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a0155330 con 0x7fd8a015c160 2024-04-22T07:04:40.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.378+0000 7fd89ffff640 1 -- 172.21.15.44:0/3230004973 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2685015860 0 0) 0x7fd894001110 con 0x7fd8a0072c20 2024-04-22T07:04:40.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.378+0000 7fd8a5983640 1 --2- 172.21.15.44:0/3230004973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a015c160 0x7fd8a015c590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00d1620 tx=0x7fd89002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db463c2ed31ce3ff server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.378+0000 7fd89ffff640 1 -- 172.21.15.44:0/3230004973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890039070 con 0x7fd8a015c160 2024-04-22T07:04:40.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.378+0000 7fd89ffff640 1 -- 172.21.15.44:0/3230004973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89002cd00 con 0x7fd8a015c160 2024-04-22T07:04:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.378+0000 7fd89ffff640 1 -- 172.21.15.44:0/3230004973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890034660 con 0x7fd8a015c160 2024-04-22T07:04:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.379+0000 7fd8a73e6640 1 -- 172.21.15.44:0/3230004973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a015c160 msgr2=0x7fd8a015c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.379+0000 7fd8a73e6640 1 --2- 172.21.15.44:0/3230004973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a015c160 0x7fd8a015c590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd8a00d1620 tx=0x7fd89002bf50 comp rx=0 tx=0).stop 2024-04-22T07:04:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.379+0000 7fd8a73e6640 1 -- 172.21.15.44:0/3230004973 shutdown_connections 2024-04-22T07:04:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.379+0000 7fd8a73e6640 1 --2- 172.21.15.44:0/3230004973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a015c160 0x7fd8a015c590 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.379+0000 7fd8a73e6640 1 --2- 172.21.15.44:0/3230004973 >> v2:172.21.15.70:3300/0 conn(0x7fd8a0154f50 0x7fd8a015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.379+0000 7fd8a73e6640 1 -- 172.21.15.44:0/3230004973 >> 172.21.15.44:0/3230004973 conn(0x7fd8a006caa0 msgr2=0x7fd8a006ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.379+0000 7fd8a73e6640 1 -- 172.21.15.44:0/3230004973 shutdown_connections 2024-04-22T07:04:40.378 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.379+0000 7fd8a73e6640 1 -- 172.21.15.44:0/3230004973 wait complete. 2024-04-22T07:04:40.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.380+0000 7fd8a73e6640 1 Processor -- start 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.380+0000 7fd8a73e6640 1 -- start start 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a73e6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a0072c20 0x7fd8a0156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a73e6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a0154f50 0x7fd8a01f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a73e6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a015c160 0x7fd8a01578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a73e6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00c5930 con 0x7fd8a0072c20 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a73e6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00c5650 con 0x7fd8a0154f50 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a73e6640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a00c5c10 con 0x7fd8a015c160 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a5182640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a0072c20 0x7fd8a0156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a5182640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a0072c20 0x7fd8a0156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41064/0 (socket says 172.21.15.44:41064) 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a5182640 1 -- 172.21.15.44:0/1630308122 learned_addr learned my addr 172.21.15.44:0/1630308122 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a4981640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a0154f50 0x7fd8a01f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a5983640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a015c160 0x7fd8a01578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a5182640 1 -- 172.21.15.44:0/1630308122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a015c160 msgr2=0x7fd8a01578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a5182640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a015c160 0x7fd8a01578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a5182640 1 -- 172.21.15.44:0/1630308122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a0154f50 msgr2=0x7fd8a01f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a5182640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a0154f50 0x7fd8a01f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.381+0000 7fd8a5182640 1 -- 172.21.15.44:0/1630308122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a01f4ba0 con 0x7fd8a0072c20 2024-04-22T07:04:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.382+0000 7fd8a5182640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a0072c20 0x7fd8a0156ac0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd89800e9e0 tx=0x7fd89800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.382+0000 7fd89dffb640 1 -- 172.21.15.44:0/1630308122 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898017070 con 0x7fd8a0072c20 2024-04-22T07:04:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.382+0000 7fd89dffb640 1 -- 172.21.15.44:0/1630308122 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89800c9d0 con 0x7fd8a0072c20 2024-04-22T07:04:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.382+0000 7fd8a73e6640 1 -- 172.21.15.44:0/1630308122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a01e42b0 con 0x7fd8a0072c20 2024-04-22T07:04:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.382+0000 7fd89dffb640 1 -- 172.21.15.44:0/1630308122 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89800cdc0 con 0x7fd8a0072c20 2024-04-22T07:04:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.382+0000 7fd8a73e6640 1 -- 172.21.15.44:0/1630308122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a01e4c30 con 0x7fd8a0072c20 2024-04-22T07:04:40.382 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.383+0000 7fd8a73e6640 1 -- 172.21.15.44:0/1630308122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a004fdc0 con 0x7fd8a0072c20 2024-04-22T07:04:40.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.384+0000 7fd89dffb640 1 -- 172.21.15.44:0/1630308122 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd89801a070 con 0x7fd8a0072c20 2024-04-22T07:04:40.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.384+0000 7fd89dffb640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd88c06c8f0 0x7fd88c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.384+0000 7fd89dffb640 1 -- 172.21.15.44:0/1630308122 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd89808bab0 con 0x7fd8a0072c20 2024-04-22T07:04:40.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.384+0000 7fd8a4981640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd88c06c8f0 0x7fd88c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.384+0000 7fd8a4981640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd88c06c8f0 0x7fd88c06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0152330 tx=0x7fd894007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.386 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.388+0000 7fd89dffb640 1 -- 172.21.15.44:0/1630308122 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8a01f4ba0 con 0x7fd8a0072c20 2024-04-22T07:04:40.557 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:04:40.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:04:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.574+0000 7fd8a73e6640 1 -- 172.21.15.44:0/1630308122 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8a00cb410 con 0x7fd8a0072c20 2024-04-22T07:04:40.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.575+0000 7fd89dffb640 1 -- 172.21.15.44:0/1630308122 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16557 (secure 0 0 0) 0x7fd898059890 con 0x7fd8a0072c20 2024-04-22T07:04:40.573 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:40.573 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":23,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:14.778262+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:40.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 -- 172.21.15.44:0/1630308122 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd88c06c8f0 msgr2=0x7fd88c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd88c06c8f0 0x7fd88c06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0152330 tx=0x7fd894007b50 comp rx=0 tx=0).stop 2024-04-22T07:04:40.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 -- 172.21.15.44:0/1630308122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a0072c20 msgr2=0x7fd8a0156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a0072c20 0x7fd8a0156ac0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd89800e9e0 tx=0x7fd89800eea0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 -- 172.21.15.44:0/1630308122 shutdown_connections 2024-04-22T07:04:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd88c06c8f0 0x7fd88c06edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd8a015c160 0x7fd8a01578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a0154f50 0x7fd8a01f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 --2- 172.21.15.44:0/1630308122 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd8a0072c20 0x7fd8a0156ac0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 -- 172.21.15.44:0/1630308122 >> 172.21.15.44:0/1630308122 conn(0x7fd8a006caa0 msgr2=0x7fd8a00cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 -- 172.21.15.44:0/1630308122 shutdown_connections 2024-04-22T07:04:40.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.577+0000 7fd8a73e6640 1 -- 172.21.15.44:0/1630308122 wait complete. 2024-04-22T07:04:40.587 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:04:01.723984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:11.652423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:12.098854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:40.587 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:40.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.754+0000 7ff00f248640 1 Processor -- start 2024-04-22T07:04:40.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.754+0000 7ff00f248640 1 -- start start 2024-04-22T07:04:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00f248640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff008151de0 0x7ff0080cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00f248640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff00815c3c0 0x7ff00815c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00f248640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff00815cd70 con 0x7ff00815b950 2024-04-22T07:04:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00f248640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7ff0080c6770 con 0x7ff008151de0 2024-04-22T07:04:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00f248640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff0080c5880 con 0x7ff00815c3c0 2024-04-22T07:04:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00cfe4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff008151de0 0x7ff0080cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00cfe4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff008151de0 0x7ff0080cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41080/0 (socket says 172.21.15.44:41080) 2024-04-22T07:04:40.753 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00cfe4640 1 -- 172.21.15.44:0/4140536473 learned_addr learned my addr 172.21.15.44:0/4140536473 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:40.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00d7e5640 1 --2- 172.21.15.44:0/4140536473 >> v2:172.21.15.70:3300/0 conn(0x7ff00815c3c0 0x7ff00815c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00cfe4640 1 -- 172.21.15.44:0/4140536473 >> v2:172.21.15.70:3300/0 conn(0x7ff00815c3c0 msgr2=0x7ff00815c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:40.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00cfe4640 1 --2- 172.21.15.44:0/4140536473 >> v2:172.21.15.70:3300/0 conn(0x7ff00815c3c0 0x7ff00815c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00cfe4640 1 -- 172.21.15.44:0/4140536473 >> v1:172.21.15.70:6789/0 conn(0x7ff00815b950 legacy=0x7ff0080676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.754 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.755+0000 7ff00cfe4640 1 -- 172.21.15.44:0/4140536473 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0080c5b60 con 0x7ff008151de0 2024-04-22T07:04:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.756+0000 7ff00cfe4640 1 --2- 172.21.15.44:0/4140536473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff008151de0 0x7ff0080cb120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff008073720 tx=0x7feffc02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ed73daa4ed98ef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.756+0000 7ff0077fe640 1 -- 172.21.15.44:0/4140536473 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc02f850 con 0x7ff008151de0 2024-04-22T07:04:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.756+0000 7ff0077fe640 1 -- 172.21.15.44:0/4140536473 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feffc02f9f0 con 0x7ff008151de0 2024-04-22T07:04:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.756+0000 7ff0077fe640 1 -- 172.21.15.44:0/4140536473 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc02fdc0 con 0x7ff008151de0 2024-04-22T07:04:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.757+0000 7ff00f248640 1 -- 172.21.15.44:0/4140536473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff008151de0 msgr2=0x7ff0080cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.757+0000 7ff00f248640 1 --2- 172.21.15.44:0/4140536473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff008151de0 0x7ff0080cb120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff008073720 tx=0x7feffc02ba30 comp rx=0 tx=0).stop 2024-04-22T07:04:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.757+0000 7ff00f248640 1 -- 172.21.15.44:0/4140536473 shutdown_connections 2024-04-22T07:04:40.761 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.757+0000 7ff00f248640 1 --2- 172.21.15.44:0/4140536473 >> v2:172.21.15.70:3300/0 conn(0x7ff00815c3c0 0x7ff00815c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.757+0000 7ff00f248640 1 --2- 172.21.15.44:0/4140536473 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff008151de0 0x7ff0080cb120 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.757+0000 7ff00f248640 1 -- 172.21.15.44:0/4140536473 >> 172.21.15.44:0/4140536473 conn(0x7ff008073da0 msgr2=0x7ff0080728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.757+0000 7ff00f248640 1 -- 172.21.15.44:0/4140536473 shutdown_connections 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.757+0000 7ff00f248640 1 -- 172.21.15.44:0/4140536473 wait complete. 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.758+0000 7ff00f248640 1 Processor -- start 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.758+0000 7ff00f248640 1 -- start start 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00f248640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff008151de0 0x7ff0080c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00f248640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff00815b950 0x7ff00814bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00f248640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff00815c3c0 0x7ff0081f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00f248640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff0080c6400 con 0x7ff00815b950 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00f248640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff0080c6120 con 0x7ff008151de0 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00f248640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff00815bd90 con 0x7ff00815c3c0 2024-04-22T07:04:40.762 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff007fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff00815b950 0x7ff00814bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00cfe4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff008151de0 0x7ff0080c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00cfe4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff008151de0 0x7ff0080c8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:54662/0 (socket says 172.21.15.44:54662) 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff007fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff00815b950 0x7ff00814bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41092/0 (socket says 172.21.15.44:41092) 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00cfe4640 1 -- 172.21.15.44:0/3643439925 learned_addr learned my addr 172.21.15.44:0/3643439925 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00d7e5640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff00815c3c0 0x7ff0081f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00cfe4640 1 -- 172.21.15.44:0/3643439925 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff00815c3c0 msgr2=0x7ff0081f0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00cfe4640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff00815c3c0 0x7ff0081f0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00cfe4640 1 -- 172.21.15.44:0/3643439925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff00815b950 msgr2=0x7ff00814bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00cfe4640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff00815b950 0x7ff00814bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00cfe4640 1 -- 172.21.15.44:0/3643439925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0080ca450 con 0x7ff008151de0 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff007fff640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff00815b950 0x7ff00814bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.759+0000 7ff00cfe4640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff008151de0 0x7ff0080c8420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feffc000c00 tx=0x7feffc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.760+0000 7ff0057fa640 1 -- 172.21.15.44:0/3643439925 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc047070 con 0x7ff008151de0 2024-04-22T07:04:40.763 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.760+0000 7ff0057fa640 1 -- 172.21.15.44:0/3643439925 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feffc037d30 con 0x7ff008151de0 2024-04-22T07:04:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.760+0000 7ff00f248640 1 -- 172.21.15.44:0/3643439925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0080ca730 con 0x7ff008151de0 2024-04-22T07:04:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.760+0000 7ff0057fa640 1 -- 172.21.15.44:0/3643439925 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc0426c0 con 0x7ff008151de0 2024-04-22T07:04:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.760+0000 7ff00f248640 1 -- 172.21.15.44:0/3643439925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0081f47a0 con 0x7ff008151de0 2024-04-22T07:04:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.761+0000 7ff00f248640 1 -- 172.21.15.44:0/3643439925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff008068b00 con 0x7ff008151de0 2024-04-22T07:04:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.761+0000 7ff0057fa640 1 -- 172.21.15.44:0/3643439925 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feffc04a070 con 0x7ff008151de0 2024-04-22T07:04:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.761+0000 7ff0057fa640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fefec06c920 0x7fefec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.762+0000 7ff0057fa640 1 -- 172.21.15.44:0/3643439925 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7feffc0adde0 con 0x7ff008151de0 2024-04-22T07:04:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.762+0000 7ff007fff640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fefec06c920 0x7fefec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.762+0000 7ff007fff640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fefec06c920 0x7fefec06ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feff4001b70 tx=0x7feff4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:40.764 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.766+0000 7ff0057fa640 1 -- 172.21.15.44:0/3643439925 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feffc033030 con 0x7ff008151de0 2024-04-22T07:04:40.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.952+0000 7ff00f248640 1 -- 172.21.15.44:0/3643439925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0080cb700 con 0x7ff008151de0 2024-04-22T07:04:40.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.953+0000 7ff0057fa640 1 -- 172.21.15.44:0/3643439925 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7feffc07bc40 con 0x7ff008151de0 2024-04-22T07:04:40.951 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 -- 172.21.15.44:0/3643439925 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fefec06c920 msgr2=0x7fefec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fefec06c920 0x7fefec06ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feff4001b70 tx=0x7feff4009040 comp rx=0 tx=0).stop 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 -- 172.21.15.44:0/3643439925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff008151de0 msgr2=0x7ff0080c8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff008151de0 0x7ff0080c8420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feffc000c00 tx=0x7feffc004730 comp rx=0 tx=0).stop 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 -- 172.21.15.44:0/3643439925 shutdown_connections 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fefec06c920 0x7fefec06ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff00815c3c0 0x7ff0081f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff00815b950 0x7ff00814bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 --2- 172.21.15.44:0/3643439925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff008151de0 0x7ff0080c8420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 -- 172.21.15.44:0/3643439925 >> 172.21.15.44:0/3643439925 conn(0x7ff008073da0 msgr2=0x7ff0080676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 -- 172.21.15.44:0/3643439925 shutdown_connections 2024-04-22T07:04:40.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:40.955+0000 7ff00f248640 1 -- 172.21.15.44:0/3643439925 wait complete. 2024-04-22T07:04:40.963 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:04:41.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.069+0000 7f94b1a14640 1 Processor -- start 2024-04-22T07:04:41.067 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.069+0000 7f94b1a14640 1 -- start start 2024-04-22T07:04:41.068 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94b1a14640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f94ac15ecd0 0x7f94ac0cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94b1a14640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94ac0c5cf0 0x7f94ac0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94b1a14640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94ac15f1a0 con 0x7f94ac154f00 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94b1a14640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f94ac0cb660 con 0x7f94ac15ecd0 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94b1a14640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f94ac0cb940 con 0x7f94ac0c5cf0 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94aa7fc640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f94ac15ecd0 0x7f94ac0cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94aa7fc640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f94ac15ecd0 0x7f94ac0cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41102/0 (socket says 172.21.15.44:41102) 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94aa7fc640 1 -- 172.21.15.44:0/2793593038 learned_addr learned my addr 172.21.15.44:0/2793593038 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94ab7fe640 1 --2- 172.21.15.44:0/2793593038 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94ac0c5cf0 0x7f94ac0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94aa7fc640 1 -- 172.21.15.44:0/2793593038 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94ac0c5cf0 msgr2=0x7f94ac0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94aa7fc640 1 --2- 172.21.15.44:0/2793593038 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94ac0c5cf0 0x7f94ac0ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94aa7fc640 1 -- 172.21.15.44:0/2793593038 >> v1:172.21.15.70:6789/0 conn(0x7f94ac154f00 legacy=0x7f94ac15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.070+0000 7f94aa7fc640 1 -- 172.21.15.44:0/2793593038 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac15aee0 con 0x7f94ac15ecd0 2024-04-22T07:04:41.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.071+0000 7f94aa7fc640 1 --2- 172.21.15.44:0/2793593038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac15ecd0 0x7f94ac0cca60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9494009870 tx=0x7f949402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8304bafdcb1017b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.071+0000 7f94a9ffb640 1 -- 172.21.15.44:0/2793593038 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494034070 con 0x7f94ac15ecd0 2024-04-22T07:04:41.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.071+0000 7f94a9ffb640 1 -- 172.21.15.44:0/2793593038 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9494030840 con 0x7f94ac15ecd0 2024-04-22T07:04:41.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.071+0000 7f94a9ffb640 1 -- 172.21.15.44:0/2793593038 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494030c10 con 0x7f94ac15ecd0 2024-04-22T07:04:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.072+0000 7f94b1a14640 1 -- 172.21.15.44:0/2793593038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac15ecd0 msgr2=0x7f94ac0cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.072+0000 7f94b1a14640 1 --2- 172.21.15.44:0/2793593038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac15ecd0 0x7f94ac0cca60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9494009870 tx=0x7f949402b980 comp rx=0 tx=0).stop 2024-04-22T07:04:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.072+0000 7f94b1a14640 1 -- 172.21.15.44:0/2793593038 shutdown_connections 2024-04-22T07:04:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.072+0000 7f94b1a14640 1 --2- 172.21.15.44:0/2793593038 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94ac0c5cf0 0x7f94ac0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.072+0000 7f94b1a14640 1 --2- 172.21.15.44:0/2793593038 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac15ecd0 0x7f94ac0cca60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.072+0000 7f94b1a14640 1 -- 172.21.15.44:0/2793593038 >> 172.21.15.44:0/2793593038 conn(0x7f94ac06d4a0 msgr2=0x7f94ac0676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.072+0000 7f94b1a14640 1 -- 172.21.15.44:0/2793593038 shutdown_connections 2024-04-22T07:04:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.072+0000 7f94b1a14640 1 -- 172.21.15.44:0/2793593038 wait complete. 2024-04-22T07:04:41.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.073+0000 7f94b1a14640 1 Processor -- start 2024-04-22T07:04:41.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.073+0000 7f94b1a14640 1 -- start start 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94b1a14640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94ac0c5cf0 0x7f94ac1f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94b1a14640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac154f00 0x7f94ac157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94b1a14640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94ac15ecd0 0x7f94ac15a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94b1a14640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac0cbdb0 con 0x7f94ac154f00 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94b1a14640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f94ac1552e0 con 0x7f94ac15ecd0 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94b1a14640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f94ac0cd4e0 con 0x7f94ac0c5cf0 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94aa7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac154f00 0x7f94ac157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94aa7fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac154f00 0x7f94ac157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41110/0 (socket says 172.21.15.44:41110) 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94aa7fc640 1 -- 172.21.15.44:0/4114411653 learned_addr learned my addr 172.21.15.44:0/4114411653 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94ab7fe640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94ac15ecd0 0x7f94ac15a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.074 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94aaffd640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94ac0c5cf0 0x7f94ac1f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94aa7fc640 1 -- 172.21.15.44:0/4114411653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94ac0c5cf0 msgr2=0x7f94ac1f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94aa7fc640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94ac0c5cf0 0x7f94ac1f14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94aa7fc640 1 -- 172.21.15.44:0/4114411653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94ac15ecd0 msgr2=0x7f94ac15a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94aa7fc640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94ac15ecd0 0x7f94ac15a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.074+0000 7f94aa7fc640 1 -- 172.21.15.44:0/4114411653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ac15e070 con 0x7f94ac154f00 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.075+0000 7f94aa7fc640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac154f00 0x7f94ac157100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f949402b9c0 tx=0x7f94940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.075+0000 7f94a3fff640 1 -- 172.21.15.44:0/4114411653 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494034040 con 0x7f94ac154f00 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.075+0000 7f94a3fff640 1 -- 172.21.15.44:0/4114411653 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9494039d90 con 0x7f94ac154f00 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.075+0000 7f94b1a14640 1 -- 172.21.15.44:0/4114411653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ac15e330 con 0x7f94ac154f00 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.075+0000 7f94a3fff640 1 -- 172.21.15.44:0/4114411653 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494043630 con 0x7f94ac154f00 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.075+0000 7f94b1a14640 1 -- 172.21.15.44:0/4114411653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ac15c2a0 con 0x7f94ac154f00 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.076+0000 7f94b1a14640 1 -- 172.21.15.44:0/4114411653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ac068b00 con 0x7f94ac154f00 2024-04-22T07:04:41.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.076+0000 7f94a3fff640 1 -- 172.21.15.44:0/4114411653 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9494037070 con 0x7f94ac154f00 2024-04-22T07:04:41.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.077+0000 7f94a3fff640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f948c06c8f0 0x7f948c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.077+0000 7f94a3fff640 1 -- 172.21.15.44:0/4114411653 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f94940adcb0 con 0x7f94ac154f00 2024-04-22T07:04:41.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.077+0000 7f94aaffd640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f948c06c8f0 0x7f948c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.077+0000 7f94aaffd640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f948c06c8f0 0x7f948c06edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94a4001780 tx=0x7f94a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.078 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.081+0000 7f94a3fff640 1 -- 172.21.15.44:0/4114411653 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f949403e070 con 0x7f94ac154f00 2024-04-22T07:04:41.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.266+0000 7f94b1a14640 1 -- 172.21.15.44:0/4114411653 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f94ac0cc0c0 con 0x7f94ac154f00 2024-04-22T07:04:41.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.267+0000 7f94a3fff640 1 -- 172.21.15.44:0/4114411653 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f949407ba90 con 0x7f94ac154f00 2024-04-22T07:04:41.264 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T07:04:41.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.268+0000 7f94b1a14640 1 -- 172.21.15.44:0/4114411653 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f948c06c8f0 msgr2=0x7f948c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.268+0000 7f94b1a14640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f948c06c8f0 0x7f948c06edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94a4001780 tx=0x7f94a4009040 comp rx=0 tx=0).stop 2024-04-22T07:04:41.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.268+0000 7f94b1a14640 1 -- 172.21.15.44:0/4114411653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac154f00 msgr2=0x7f94ac157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.268+0000 7f94b1a14640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac154f00 0x7f94ac157100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f949402b9c0 tx=0x7f94940046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.268+0000 7f94b1a14640 1 -- 172.21.15.44:0/4114411653 shutdown_connections 2024-04-22T07:04:41.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.268+0000 7f94b1a14640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f948c06c8f0 0x7f948c06edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.268+0000 7f94b1a14640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94ac15ecd0 0x7f94ac15a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.268+0000 7f94b1a14640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f94ac154f00 0x7f94ac157100 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.268+0000 7f94b1a14640 1 --2- 172.21.15.44:0/4114411653 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f94ac0c5cf0 0x7f94ac1f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.268+0000 7f94b1a14640 1 -- 172.21.15.44:0/4114411653 >> 172.21.15.44:0/4114411653 conn(0x7f94ac06d4a0 msgr2=0x7f94ac0ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.269+0000 7f94b1a14640 1 -- 172.21.15.44:0/4114411653 shutdown_connections 2024-04-22T07:04:41.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.269+0000 7f94b1a14640 1 -- 172.21.15.44:0/4114411653 wait complete. 2024-04-22T07:04:41.277 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:04:41.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.383+0000 7f506c3b2640 1 Processor -- start 2024-04-22T07:04:41.381 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.383+0000 7f506c3b2640 1 -- start start 2024-04-22T07:04:41.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.384+0000 7f506c3b2640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f50641552c0 0x7f50640b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.384+0000 7f506c3b2640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50641556a0 con 0x7f50640cc920 2024-04-22T07:04:41.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.384+0000 7f506c3b2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f506415b160 con 0x7f50640caec0 2024-04-22T07:04:41.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.384+0000 7f506c3b2640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f506415b440 con 0x7f50641552c0 2024-04-22T07:04:41.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.385+0000 7f506a14e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f50641552c0 0x7f50640b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.385+0000 7f506a94f640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f50640cc920 0x7f50640cd310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55650/0 (socket says 172.21.15.44:55650) 2024-04-22T07:04:41.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.385+0000 7f506a94f640 1 -- 172.21.15.44:0/2498024855 learned_addr learned my addr 172.21.15.44:0/2498024855 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:41.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.385+0000 7f506a14e640 1 -- 172.21.15.44:0/2498024855 >> v1:172.21.15.70:6789/0 conn(0x7f50640caec0 legacy=0x7f50640d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.385+0000 7f506a14e640 1 -- 172.21.15.44:0/2498024855 >> v1:172.21.15.44:6789/0 conn(0x7f50640cc920 legacy=0x7f50640cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.383 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.385+0000 7f506914c640 1 -- 172.21.15.44:0/2498024855 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 968221249 0 0) 0x7f50641556a0 con 0x7f50640cc920 2024-04-22T07:04:41.384 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.385+0000 7f506a14e640 1 -- 172.21.15.44:0/2498024855 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506415b720 con 0x7f50641552c0 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506a14e640 1 --2- 172.21.15.44:0/2498024855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50641552c0 0x7f50640b32a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5060009d90 tx=0x7f506002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67e6aa87122dd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506914c640 1 -- 172.21.15.44:0/2498024855 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060039070 con 0x7f50641552c0 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506914c640 1 -- 172.21.15.44:0/2498024855 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506002cd00 con 0x7f50641552c0 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506914c640 1 -- 172.21.15.44:0/2498024855 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060034660 con 0x7f50641552c0 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506c3b2640 1 -- 172.21.15.44:0/2498024855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50641552c0 msgr2=0x7f50640b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506c3b2640 1 --2- 172.21.15.44:0/2498024855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50641552c0 0x7f50640b32a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5060009d90 tx=0x7f506002bf50 comp rx=0 tx=0).stop 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506c3b2640 1 -- 172.21.15.44:0/2498024855 shutdown_connections 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506c3b2640 1 --2- 172.21.15.44:0/2498024855 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50641552c0 0x7f50640b32a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506c3b2640 1 -- 172.21.15.44:0/2498024855 >> 172.21.15.44:0/2498024855 conn(0x7f50640676b0 msgr2=0x7f5064067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506c3b2640 1 -- 172.21.15.44:0/2498024855 shutdown_connections 2024-04-22T07:04:41.385 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.386+0000 7f506c3b2640 1 -- 172.21.15.44:0/2498024855 wait complete. 2024-04-22T07:04:41.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506c3b2640 1 Processor -- start 2024-04-22T07:04:41.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506c3b2640 1 -- start start 2024-04-22T07:04:41.387 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506c3b2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50640caec0 0x7f50641562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506c3b2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50640cc920 0x7f50641f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506c3b2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50641552c0 0x7f50640cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506c3b2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f50640c6a00 con 0x7f50640cc920 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506c3b2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f506415bce0 con 0x7f50640caec0 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506c3b2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f50640d1130 con 0x7f50641552c0 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506994d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50640cc920 0x7f50641f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506994d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50640cc920 0x7f50641f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41130/0 (socket says 172.21.15.44:41130) 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506994d640 1 -- 172.21.15.44:0/218651668 learned_addr learned my addr 172.21.15.44:0/218651668 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506a94f640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50641552c0 0x7f50640cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.388+0000 7f506a14e640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50640caec0 0x7f50641562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f506994d640 1 -- 172.21.15.44:0/218651668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50641552c0 msgr2=0x7f50640cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f506994d640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50641552c0 0x7f50640cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f506994d640 1 -- 172.21.15.44:0/218651668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50640caec0 msgr2=0x7f50641562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.388 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f506994d640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50640caec0 0x7f50641562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f506994d640 1 -- 172.21.15.44:0/218651668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50641f08b0 con 0x7f50640cc920 2024-04-22T07:04:41.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f506a14e640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50640caec0 0x7f50641562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:41.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f506994d640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50640cc920 0x7f50641f3e20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f505400c430 tx=0x7f505400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f505affd640 1 -- 172.21.15.44:0/218651668 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054017070 con 0x7f50640cc920 2024-04-22T07:04:41.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f505affd640 1 -- 172.21.15.44:0/218651668 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50540052d0 con 0x7f50640cc920 2024-04-22T07:04:41.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f505affd640 1 -- 172.21.15.44:0/218651668 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054003830 con 0x7f50640cc920 2024-04-22T07:04:41.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f506c3b2640 1 -- 172.21.15.44:0/218651668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506414eff0 con 0x7f50640cc920 2024-04-22T07:04:41.389 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.389+0000 7f506c3b2640 1 -- 172.21.15.44:0/218651668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50641f0b70 con 0x7f50640cc920 2024-04-22T07:04:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.390+0000 7f506c3b2640 1 -- 172.21.15.44:0/218651668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5064068b00 con 0x7f50640cc920 2024-04-22T07:04:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.391+0000 7f505affd640 1 -- 172.21.15.44:0/218651668 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f505401a070 con 0x7f50640cc920 2024-04-22T07:04:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.391+0000 7f505affd640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f504806c920 0x7f504806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.391+0000 7f505affd640 1 -- 172.21.15.44:0/218651668 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f505408ade0 con 0x7f50640cc920 2024-04-22T07:04:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.391+0000 7f506a14e640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f504806c920 0x7f504806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.390 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.392+0000 7f506a14e640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f504806c920 0x7f504806ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5060036000 tx=0x7f5060030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.393 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.395+0000 7f505affd640 1 -- 172.21.15.44:0/218651668 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5054058bc0 con 0x7f50640cc920 2024-04-22T07:04:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.582+0000 7f506c3b2640 1 -- 172.21.15.44:0/218651668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f50640ca6f0 con 0x7f50640cc920 2024-04-22T07:04:41.580 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.582+0000 7f505affd640 1 -- 172.21.15.44:0/218651668 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f505405cec0 con 0x7f50640cc920 2024-04-22T07:04:41.580 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T07:04:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.584+0000 7f506c3b2640 1 -- 172.21.15.44:0/218651668 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f504806c920 msgr2=0x7f504806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.584+0000 7f506c3b2640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f504806c920 0x7f504806ede0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5060036000 tx=0x7f5060030000 comp rx=0 tx=0).stop 2024-04-22T07:04:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.584+0000 7f506c3b2640 1 -- 172.21.15.44:0/218651668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50640cc920 msgr2=0x7f50641f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.584+0000 7f506c3b2640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50640cc920 0x7f50641f3e20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f505400c430 tx=0x7f505400c8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.584+0000 7f506c3b2640 1 -- 172.21.15.44:0/218651668 shutdown_connections 2024-04-22T07:04:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.584+0000 7f506c3b2640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f504806c920 0x7f504806ede0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.584+0000 7f506c3b2640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f50641552c0 0x7f50640cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.584+0000 7f506c3b2640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f50640cc920 0x7f50641f3e20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.584+0000 7f506c3b2640 1 --2- 172.21.15.44:0/218651668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50640caec0 0x7f50641562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.584+0000 7f506c3b2640 1 -- 172.21.15.44:0/218651668 >> 172.21.15.44:0/218651668 conn(0x7f50640676b0 msgr2=0x7f50640cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.585+0000 7f506c3b2640 1 -- 172.21.15.44:0/218651668 shutdown_connections 2024-04-22T07:04:41.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.585+0000 7f506c3b2640 1 -- 172.21.15.44:0/218651668 wait complete. 2024-04-22T07:04:41.593 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.695+0000 7f7d44d54640 1 Processor -- start 2024-04-22T07:04:41.693 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.695+0000 7f7d44d54640 1 -- start start 2024-04-22T07:04:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d44d54640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7d40154f50 0x7f7d4015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d44d54640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400ccf30 0x7f7d400cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d44d54640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d4015bc20 con 0x7f7d400cc880 2024-04-22T07:04:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d44d54640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f7d4015f1b0 con 0x7f7d40154f50 2024-04-22T07:04:41.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d44d54640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7d400cb7c0 con 0x7f7d400ccf30 2024-04-22T07:04:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d3e59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7d40154f50 0x7f7d4015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d3e59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7d40154f50 0x7f7d4015b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41140/0 (socket says 172.21.15.44:41140) 2024-04-22T07:04:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d3e59c640 1 -- 172.21.15.44:0/1929059893 learned_addr learned my addr 172.21.15.44:0/1929059893 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d3ed9d640 1 --2- 172.21.15.44:0/1929059893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400ccf30 0x7f7d400cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d3d59a640 1 -- 172.21.15.44:0/1929059893 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2716530070 0 0) 0x7f7d28001110 con 0x7f7d400cc880 2024-04-22T07:04:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d3e59c640 1 -- 172.21.15.44:0/1929059893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400ccf30 msgr2=0x7f7d400cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d3e59c640 1 --2- 172.21.15.44:0/1929059893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400ccf30 0x7f7d400cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d3e59c640 1 -- 172.21.15.44:0/1929059893 >> v1:172.21.15.70:6789/0 conn(0x7f7d400cc880 legacy=0x7f7d400c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.696+0000 7f7d3e59c640 1 -- 172.21.15.44:0/1929059893 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4015e3f0 con 0x7f7d40154f50 2024-04-22T07:04:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.697+0000 7f7d3e59c640 1 --2- 172.21.15.44:0/1929059893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 0x7f7d4015b6e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009870 tx=0x7f7d3802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56905c84e514b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.697+0000 7f7d3d59a640 1 -- 172.21.15.44:0/1929059893 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3802f840 con 0x7f7d40154f50 2024-04-22T07:04:41.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.697+0000 7f7d3d59a640 1 -- 172.21.15.44:0/1929059893 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d3802f9e0 con 0x7f7d40154f50 2024-04-22T07:04:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.697+0000 7f7d3d59a640 1 -- 172.21.15.44:0/1929059893 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3802fdb0 con 0x7f7d40154f50 2024-04-22T07:04:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.698+0000 7f7d44d54640 1 -- 172.21.15.44:0/1929059893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 msgr2=0x7f7d4015b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.698+0000 7f7d44d54640 1 --2- 172.21.15.44:0/1929059893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 0x7f7d4015b6e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009870 tx=0x7f7d3802ba20 comp rx=0 tx=0).stop 2024-04-22T07:04:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.698+0000 7f7d44d54640 1 -- 172.21.15.44:0/1929059893 shutdown_connections 2024-04-22T07:04:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.698+0000 7f7d44d54640 1 --2- 172.21.15.44:0/1929059893 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400ccf30 0x7f7d400cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.698+0000 7f7d44d54640 1 --2- 172.21.15.44:0/1929059893 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 0x7f7d4015b6e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.698+0000 7f7d44d54640 1 -- 172.21.15.44:0/1929059893 >> 172.21.15.44:0/1929059893 conn(0x7f7d40067d40 msgr2=0x7f7d4006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.698+0000 7f7d44d54640 1 -- 172.21.15.44:0/1929059893 shutdown_connections 2024-04-22T07:04:41.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.698+0000 7f7d44d54640 1 -- 172.21.15.44:0/1929059893 wait complete. 2024-04-22T07:04:41.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.699+0000 7f7d44d54640 1 Processor -- start 2024-04-22T07:04:41.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.699+0000 7f7d44d54640 1 -- start start 2024-04-22T07:04:41.699 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d44d54640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400cc880 0x7f7d4015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d44d54640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400ccf30 0x7f7d40153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d44d54640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 0x7f7d401f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d44d54640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4015ec90 con 0x7f7d40154f50 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d44d54640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4015e9b0 con 0x7f7d400cc880 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d44d54640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7d40155330 con 0x7f7d400ccf30 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d3ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 0x7f7d401f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d3ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 0x7f7d401f0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41156/0 (socket says 172.21.15.44:41156) 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d3ed9d640 1 -- 172.21.15.44:0/1539671391 learned_addr learned my addr 172.21.15.44:0/1539671391 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d3e59c640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400cc880 0x7f7d4015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.700+0000 7f7d3dd9b640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400ccf30 0x7f7d40153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d3ed9d640 1 -- 172.21.15.44:0/1539671391 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400ccf30 msgr2=0x7f7d40153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d3ed9d640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400ccf30 0x7f7d40153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d3ed9d640 1 -- 172.21.15.44:0/1539671391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400cc880 msgr2=0x7f7d4015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.700 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d3ed9d640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400cc880 0x7f7d4015d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d3ed9d640 1 -- 172.21.15.44:0/1539671391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d401511b0 con 0x7f7d40154f50 2024-04-22T07:04:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d3e59c640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400cc880 0x7f7d4015d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d3ed9d640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 0x7f7d401f0d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000efc0 tx=0x7f7d3000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d36ffd640 1 -- 172.21.15.44:0/1539671391 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30018020 con 0x7f7d40154f50 2024-04-22T07:04:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d36ffd640 1 -- 172.21.15.44:0/1539671391 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d30014480 con 0x7f7d40154f50 2024-04-22T07:04:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d44d54640 1 -- 172.21.15.44:0/1539671391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4015dd90 con 0x7f7d40154f50 2024-04-22T07:04:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d36ffd640 1 -- 172.21.15.44:0/1539671391 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30014870 con 0x7f7d40154f50 2024-04-22T07:04:41.701 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.701+0000 7f7d44d54640 1 -- 172.21.15.44:0/1539671391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d401e62c0 con 0x7f7d40154f50 2024-04-22T07:04:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.702+0000 7f7d44d54640 1 -- 172.21.15.44:0/1539671391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d400b4f90 con 0x7f7d40154f50 2024-04-22T07:04:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.702+0000 7f7d36ffd640 1 -- 172.21.15.44:0/1539671391 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d30005d10 con 0x7f7d40154f50 2024-04-22T07:04:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.703+0000 7f7d36ffd640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7d2006c8f0 0x7f7d2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.703+0000 7f7d36ffd640 1 -- 172.21.15.44:0/1539671391 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7d3008c500 con 0x7f7d40154f50 2024-04-22T07:04:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.703+0000 7f7d3e59c640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7d2006c8f0 0x7f7d2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:41.702 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.703+0000 7f7d3e59c640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7d2006c8f0 0x7f7d2006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7d38036000 tx=0x7f7d38008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:41.705 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.707+0000 7f7d36ffd640 1 -- 172.21.15.44:0/1539671391 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d400b4f90 con 0x7f7d40154f50 2024-04-22T07:04:41.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.932+0000 7f7d44d54640 1 -- 172.21.15.44:0/1539671391 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d4014efd0 con 0x7f7d40154f50 2024-04-22T07:04:41.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.932+0000 7f7d36ffd640 1 -- 172.21.15.44:0/1539671391 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6904 (secure 0 0 0) 0x7f7d30022070 con 0x7f7d40154f50 2024-04-22T07:04:41.930 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:41.930 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:04:39.933702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4324,"mds_1":4303,"mds_2":4300,"mds_3":4278,"mds_4":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"b","rank":3,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4324,"qdb_cluster":[4324,4303,4300,4278]},"id":1}]} 2024-04-22T07:04:41.931 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 16 2024-04-22T07:04:41.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 -- 172.21.15.44:0/1539671391 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7d2006c8f0 msgr2=0x7f7d2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7d2006c8f0 0x7f7d2006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7d38036000 tx=0x7f7d38008030 comp rx=0 tx=0).stop 2024-04-22T07:04:41.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 -- 172.21.15.44:0/1539671391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 msgr2=0x7f7d401f0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:41.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 0x7f7d401f0d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000efc0 tx=0x7f7d3000c490 comp rx=0 tx=0).stop 2024-04-22T07:04:41.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 -- 172.21.15.44:0/1539671391 shutdown_connections 2024-04-22T07:04:41.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7d2006c8f0 0x7f7d2006edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7d400ccf30 0x7f7d40153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7d40154f50 0x7f7d401f0d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 --2- 172.21.15.44:0/1539671391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d400cc880 0x7f7d4015d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:41.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 -- 172.21.15.44:0/1539671391 >> 172.21.15.44:0/1539671391 conn(0x7f7d40067d40 msgr2=0x7f7d400b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:41.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 -- 172.21.15.44:0/1539671391 shutdown_connections 2024-04-22T07:04:41.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:41.934+0000 7f7d44d54640 1 -- 172.21.15.44:0/1539671391 wait complete. 2024-04-22T07:04:41.944 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:42.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.046+0000 7f9ecdac1640 1 Processor -- start 2024-04-22T07:04:42.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.046+0000 7f9ecdac1640 1 -- start start 2024-04-22T07:04:42.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.047+0000 7f9ecdac1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9ec81561d0 0x7f9ec80c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.047+0000 7f9ecdac1640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec80c91a0 con 0x7f9ec80b3020 2024-04-22T07:04:42.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.047+0000 7f9ecdac1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec80701b0 con 0x7f9ec80ce340 2024-04-22T07:04:42.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.047+0000 7f9ecdac1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec8070490 con 0x7f9ec81561d0 2024-04-22T07:04:42.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.047+0000 7f9ec6ffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9ec80b3020 0x7f9ec80ca5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55660/0 (socket says 172.21.15.44:55660) 2024-04-22T07:04:42.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.047+0000 7f9ec6ffd640 1 -- 172.21.15.44:0/1730729455 learned_addr learned my addr 172.21.15.44:0/1730729455 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:42.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.047+0000 7f9ec77fe640 1 --2- 172.21.15.44:0/1730729455 >> v2:172.21.15.70:3300/0 conn(0x7f9ec81561d0 0x7f9ec80c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.047+0000 7f9ec67fc640 1 -- 172.21.15.44:0/1730729455 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2335597750 0 0) 0x7f9ec80c91a0 con 0x7f9ec80b3020 2024-04-22T07:04:42.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.048+0000 7f9ec67fc640 1 -- 172.21.15.44:0/1730729455 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9eb0003520 con 0x7f9ec80b3020 2024-04-22T07:04:42.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.048+0000 7f9ec77fe640 1 -- 172.21.15.44:0/1730729455 >> v1:172.21.15.70:6789/0 conn(0x7f9ec80ce340 legacy=0x7f9ec8163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.048+0000 7f9ec77fe640 1 -- 172.21.15.44:0/1730729455 >> v1:172.21.15.44:6789/0 conn(0x7f9ec80b3020 legacy=0x7f9ec80ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.048+0000 7f9ec77fe640 1 -- 172.21.15.44:0/1730729455 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec8070770 con 0x7f9ec81561d0 2024-04-22T07:04:42.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.048+0000 7f9ec77fe640 1 --2- 172.21.15.44:0/1730729455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec81561d0 0x7f9ec80c5240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80097a0 tx=0x7f9eb802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d382d7de184c699e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.049+0000 7f9ec67fc640 1 -- 172.21.15.44:0/1730729455 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8034070 con 0x7f9ec81561d0 2024-04-22T07:04:42.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.049+0000 7f9ec67fc640 1 -- 172.21.15.44:0/1730729455 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb8030610 con 0x7f9ec81561d0 2024-04-22T07:04:42.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.049+0000 7f9ec67fc640 1 -- 172.21.15.44:0/1730729455 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb80309c0 con 0x7f9ec81561d0 2024-04-22T07:04:42.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.049+0000 7f9ecdac1640 1 -- 172.21.15.44:0/1730729455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec81561d0 msgr2=0x7f9ec80c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.049+0000 7f9ecdac1640 1 --2- 172.21.15.44:0/1730729455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec81561d0 0x7f9ec80c5240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80097a0 tx=0x7f9eb802b960 comp rx=0 tx=0).stop 2024-04-22T07:04:42.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.049+0000 7f9ecdac1640 1 -- 172.21.15.44:0/1730729455 shutdown_connections 2024-04-22T07:04:42.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.049+0000 7f9ecdac1640 1 --2- 172.21.15.44:0/1730729455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec81561d0 0x7f9ec80c5240 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.049+0000 7f9ecdac1640 1 -- 172.21.15.44:0/1730729455 >> 172.21.15.44:0/1730729455 conn(0x7f9ec8067ce0 msgr2=0x7f9ec80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.049+0000 7f9ecdac1640 1 -- 172.21.15.44:0/1730729455 shutdown_connections 2024-04-22T07:04:42.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.049+0000 7f9ecdac1640 1 -- 172.21.15.44:0/1730729455 wait complete. 2024-04-22T07:04:42.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.050+0000 7f9ecdac1640 1 Processor -- start 2024-04-22T07:04:42.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.050+0000 7f9ecdac1640 1 -- start start 2024-04-22T07:04:42.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ecdac1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80b3020 0x7f9ec806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ecdac1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ec80ce340 0x7f9ec80cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ecdac1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec81561d0 0x7f9ec81e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ecdac1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec80c8dd0 con 0x7f9ec81561d0 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ecdac1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec81565b0 con 0x7f9ec80b3020 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ecdac1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec80c8f50 con 0x7f9ec80ce340 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ec77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80b3020 0x7f9ec806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ec7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec81561d0 0x7f9ec81e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ec77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80b3020 0x7f9ec806d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:54706/0 (socket says 172.21.15.44:54706) 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ec7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec81561d0 0x7f9ec81e3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41160/0 (socket says 172.21.15.44:41160) 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ec77fe640 1 -- 172.21.15.44:0/4126724927 learned_addr learned my addr 172.21.15.44:0/4126724927 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.051+0000 7f9ec6ffd640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ec80ce340 0x7f9ec80cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9ec77fe640 1 -- 172.21.15.44:0/4126724927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ec80ce340 msgr2=0x7f9ec80cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9ec77fe640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ec80ce340 0x7f9ec80cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9ec77fe640 1 -- 172.21.15.44:0/4126724927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec81561d0 msgr2=0x7f9ec81e3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9ec77fe640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec81561d0 0x7f9ec81e3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9ec77fe640 1 -- 172.21.15.44:0/4126724927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec815b510 con 0x7f9ec80b3020 2024-04-22T07:04:42.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9ec7fff640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec81561d0 0x7f9ec81e3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:42.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9ec77fe640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80b3020 0x7f9ec806d330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8000c00 tx=0x7f9eb80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9eaffff640 1 -- 172.21.15.44:0/4126724927 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8047070 con 0x7f9ec80b3020 2024-04-22T07:04:42.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9eaffff640 1 -- 172.21.15.44:0/4126724927 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb8030410 con 0x7f9ec80b3020 2024-04-22T07:04:42.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9ecdac1640 1 -- 172.21.15.44:0/4126724927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec806bd90 con 0x7f9ec80b3020 2024-04-22T07:04:42.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9eaffff640 1 -- 172.21.15.44:0/4126724927 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8034040 con 0x7f9ec80b3020 2024-04-22T07:04:42.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.052+0000 7f9ecdac1640 1 -- 172.21.15.44:0/4126724927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec815b7b0 con 0x7f9ec80b3020 2024-04-22T07:04:42.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.053+0000 7f9ecdac1640 1 -- 172.21.15.44:0/4126724927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec815bbf0 con 0x7f9ec80b3020 2024-04-22T07:04:42.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.053+0000 7f9eaffff640 1 -- 172.21.15.44:0/4126724927 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9eb8037070 con 0x7f9ec80b3020 2024-04-22T07:04:42.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.054+0000 7f9eaffff640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9eb406c8f0 0x7f9eb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.054+0000 7f9eaffff640 1 -- 172.21.15.44:0/4126724927 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9eb80ad110 con 0x7f9ec80b3020 2024-04-22T07:04:42.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.054+0000 7f9ec6ffd640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9eb406c8f0 0x7f9eb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.054+0000 7f9ec6ffd640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9eb406c8f0 0x7f9eb406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8154a40 tx=0x7f9ebc007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.058+0000 7f9eaffff640 1 -- 172.21.15.44:0/4126724927 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9eb807ae70 con 0x7f9ec80b3020 2024-04-22T07:04:42.243 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.245+0000 7f9ecdac1640 1 -- 172.21.15.44:0/4126724927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ec80c9dd0 con 0x7f9ec80b3020 2024-04-22T07:04:42.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.247+0000 7f9eaffff640 1 -- 172.21.15.44:0/4126724927 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16557 (secure 0 0 0) 0x7f9eb807f170 con 0x7f9ec80b3020 2024-04-22T07:04:42.245 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:42.245 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":23,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:14.778262+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:42.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 -- 172.21.15.44:0/4126724927 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9eb406c8f0 msgr2=0x7f9eb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9eb406c8f0 0x7f9eb406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8154a40 tx=0x7f9ebc007d30 comp rx=0 tx=0).stop 2024-04-22T07:04:42.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 -- 172.21.15.44:0/4126724927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80b3020 msgr2=0x7f9ec806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80b3020 0x7f9ec806d330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8000c00 tx=0x7f9eb80046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 -- 172.21.15.44:0/4126724927 shutdown_connections 2024-04-22T07:04:42.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9eb406c8f0 0x7f9eb406edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9ec80ce340 0x7f9ec80cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9ec81561d0 0x7f9ec81e3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.247 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 --2- 172.21.15.44:0/4126724927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80b3020 0x7f9ec806d330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 -- 172.21.15.44:0/4126724927 >> 172.21.15.44:0/4126724927 conn(0x7f9ec8067ce0 msgr2=0x7f9ec80ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 -- 172.21.15.44:0/4126724927 shutdown_connections 2024-04-22T07:04:42.248 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.249+0000 7f9ecdac1640 1 -- 172.21.15.44:0/4126724927 wait complete. 2024-04-22T07:04:42.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:04:01.723984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:11.652423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:12.098854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:42.258 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:42.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.363+0000 7f8b10d4b640 1 Processor -- start 2024-04-22T07:04:42.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.363+0000 7f8b10d4b640 1 -- start start 2024-04-22T07:04:42.362 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.364+0000 7f8b10d4b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8b0c1552c0 0x7f8b0c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.364+0000 7f8b10d4b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c0cc920 0x7f8b0c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.364+0000 7f8b10d4b640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b0c1556a0 con 0x7f8b0c0caec0 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.364+0000 7f8b10d4b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f8b0c15b160 con 0x7f8b0c1552c0 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.364+0000 7f8b10d4b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8b0c15b440 con 0x7f8b0c0cc920 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.364+0000 7f8b09d9b640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f8b0c0caec0 0x7f8b0c0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55674/0 (socket says 172.21.15.44:55674) 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.364+0000 7f8b09d9b640 1 -- 172.21.15.44:0/2820080446 learned_addr learned my addr 172.21.15.44:0/2820080446 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.364+0000 7f8b0a59c640 1 --2- 172.21.15.44:0/2820080446 >> v2:172.21.15.70:3300/0 conn(0x7f8b0c1552c0 0x7f8b0c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.364+0000 7f8b0ad9d640 1 --2- 172.21.15.44:0/2820080446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c0cc920 0x7f8b0c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.365+0000 7f8b0959a640 1 -- 172.21.15.44:0/2820080446 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2229249076 0 0) 0x7f8b0c1556a0 con 0x7f8b0c0caec0 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.365+0000 7f8b0959a640 1 -- 172.21.15.44:0/2820080446 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8af4003520 con 0x7f8b0c0caec0 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.365+0000 7f8b0a59c640 1 -- 172.21.15.44:0/2820080446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c0cc920 msgr2=0x7f8b0c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.365+0000 7f8b0a59c640 1 --2- 172.21.15.44:0/2820080446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c0cc920 0x7f8b0c0cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.365+0000 7f8b0a59c640 1 -- 172.21.15.44:0/2820080446 >> v1:172.21.15.44:6789/0 conn(0x7f8b0c0caec0 legacy=0x7f8b0c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.365+0000 7f8b0a59c640 1 -- 172.21.15.44:0/2820080446 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c15b720 con 0x7f8b0c1552c0 2024-04-22T07:04:42.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.365+0000 7f8b0ad9d640 1 --2- 172.21.15.44:0/2820080446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c0cc920 0x7f8b0c0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.365+0000 7f8b0a59c640 1 --2- 172.21.15.44:0/2820080446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b0c1552c0 0x7f8b0c0b32a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009d90 tx=0x7f8afc02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c44e77a5e5eba030 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b0959a640 1 -- 172.21.15.44:0/2820080446 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc039070 con 0x7f8b0c1552c0 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b0959a640 1 -- 172.21.15.44:0/2820080446 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8afc02cc50 con 0x7f8b0c1552c0 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b0959a640 1 -- 172.21.15.44:0/2820080446 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc034440 con 0x7f8b0c1552c0 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b10d4b640 1 -- 172.21.15.44:0/2820080446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b0c1552c0 msgr2=0x7f8b0c0b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b10d4b640 1 --2- 172.21.15.44:0/2820080446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b0c1552c0 0x7f8b0c0b32a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009d90 tx=0x7f8afc02bea0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b10d4b640 1 -- 172.21.15.44:0/2820080446 shutdown_connections 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b10d4b640 1 --2- 172.21.15.44:0/2820080446 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c0cc920 0x7f8b0c0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b10d4b640 1 --2- 172.21.15.44:0/2820080446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b0c1552c0 0x7f8b0c0b32a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b10d4b640 1 -- 172.21.15.44:0/2820080446 >> 172.21.15.44:0/2820080446 conn(0x7f8b0c0676b0 msgr2=0x7f8b0c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b10d4b640 1 -- 172.21.15.44:0/2820080446 shutdown_connections 2024-04-22T07:04:42.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.366+0000 7f8b10d4b640 1 -- 172.21.15.44:0/2820080446 wait complete. 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b10d4b640 1 Processor -- start 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b10d4b640 1 -- start start 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b10d4b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b0c0caec0 0x7f8b0c0c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b10d4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b0c0cc920 0x7f8b0c1f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b10d4b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c1552c0 0x7f8b0c15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b10d4b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c0c6a00 con 0x7f8b0c0caec0 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b10d4b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c15bce0 con 0x7f8b0c0cc920 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b10d4b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8b0c0d1130 con 0x7f8b0c1552c0 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b0a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b0c0caec0 0x7f8b0c0c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b09d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b0c0cc920 0x7f8b0c1f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b0a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b0c0caec0 0x7f8b0c0c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41176/0 (socket says 172.21.15.44:41176) 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b0a59c640 1 -- 172.21.15.44:0/4136177741 learned_addr learned my addr 172.21.15.44:0/4136177741 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.368+0000 7f8b0ad9d640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c1552c0 0x7f8b0c15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8b0a59c640 1 -- 172.21.15.44:0/4136177741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c1552c0 msgr2=0x7f8b0c15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8b0a59c640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c1552c0 0x7f8b0c15a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8b0a59c640 1 -- 172.21.15.44:0/4136177741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b0c0cc920 msgr2=0x7f8b0c1f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8b0a59c640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b0c0cc920 0x7f8b0c1f4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8b0a59c640 1 -- 172.21.15.44:0/4136177741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c156100 con 0x7f8b0c0caec0 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8b09d9b640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b0c0cc920 0x7f8b0c1f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8b0a59c640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b0c0caec0 0x7f8b0c0c95d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009eb0 tx=0x7f8afc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8afaffd640 1 -- 172.21.15.44:0/4136177741 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc039050 con 0x7f8b0c0caec0 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8afaffd640 1 -- 172.21.15.44:0/4136177741 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8afc045070 con 0x7f8b0c0caec0 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8b10d4b640 1 -- 172.21.15.44:0/4136177741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c0c9c80 con 0x7f8b0c0caec0 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8afaffd640 1 -- 172.21.15.44:0/4136177741 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc009450 con 0x7f8b0c0caec0 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.369+0000 7f8b10d4b640 1 -- 172.21.15.44:0/4136177741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c1f2110 con 0x7f8b0c0caec0 2024-04-22T07:04:42.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.370+0000 7f8b10d4b640 1 -- 172.21.15.44:0/4136177741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0c072660 con 0x7f8b0c0caec0 2024-04-22T07:04:42.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.371+0000 7f8afaffd640 1 -- 172.21.15.44:0/4136177741 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8afc03e610 con 0x7f8b0c0caec0 2024-04-22T07:04:42.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.371+0000 7f8afaffd640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8aec06c8f0 0x7f8aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.371+0000 7f8b09d9b640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8aec06c8f0 0x7f8aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.371+0000 7f8afaffd640 1 -- 172.21.15.44:0/4136177741 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8afc0ae520 con 0x7f8b0c0caec0 2024-04-22T07:04:42.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.371+0000 7f8b09d9b640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8aec06c8f0 0x7f8aec06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c1f4170 tx=0x7f8b00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.375+0000 7f8afaffd640 1 -- 172.21.15.44:0/4136177741 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8b0c072660 con 0x7f8b0c0caec0 2024-04-22T07:04:42.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.560+0000 7f8b10d4b640 1 -- 172.21.15.44:0/4136177741 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8b0c0b3800 con 0x7f8b0c0caec0 2024-04-22T07:04:42.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.561+0000 7f8afaffd640 1 -- 172.21.15.44:0/4136177741 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f8b0c156100 con 0x7f8b0c0caec0 2024-04-22T07:04:42.559 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T07:04:42.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 -- 172.21.15.44:0/4136177741 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8aec06c8f0 msgr2=0x7f8aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8aec06c8f0 0x7f8aec06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c1f4170 tx=0x7f8b00009040 comp rx=0 tx=0).stop 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 -- 172.21.15.44:0/4136177741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b0c0caec0 msgr2=0x7f8b0c0c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b0c0caec0 0x7f8b0c0c95d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009eb0 tx=0x7f8afc004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 -- 172.21.15.44:0/4136177741 shutdown_connections 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8aec06c8f0 0x7f8aec06edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8b0c1552c0 0x7f8b0c15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b0c0cc920 0x7f8b0c1f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 --2- 172.21.15.44:0/4136177741 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8b0c0caec0 0x7f8b0c0c95d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 -- 172.21.15.44:0/4136177741 >> 172.21.15.44:0/4136177741 conn(0x7f8b0c0676b0 msgr2=0x7f8b0c0cfc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 -- 172.21.15.44:0/4136177741 shutdown_connections 2024-04-22T07:04:42.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.563+0000 7f8b10d4b640 1 -- 172.21.15.44:0/4136177741 wait complete. 2024-04-22T07:04:42.571 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:04:42.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.677+0000 7fdaf0243640 1 Processor -- start 2024-04-22T07:04:42.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.677+0000 7fdaf0243640 1 -- start start 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaf0243640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 0x7fdae8072c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaf0243640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdae80cbcc0 0x7fdae80d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaf0243640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdae815af90 con 0x7fdae80d0e10 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaf0243640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdae80cb270 con 0x7fdae80cbcc0 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaf0243640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdae815b270 con 0x7fdae8155040 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaed7de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdae80cbcc0 0x7fdae80d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaedfdf640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 0x7fdae8072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaedfdf640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 0x7fdae8072c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:38270/0 (socket says 172.21.15.44:38270) 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaed7de640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdae80cbcc0 0x7fdae80d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41192/0 (socket says 172.21.15.44:41192) 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaedfdf640 1 -- 172.21.15.44:0/2914741792 learned_addr learned my addr 172.21.15.44:0/2914741792 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:42.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaee7e0640 1 --1- 172.21.15.44:0/2914741792 >> v1:172.21.15.70:6789/0 conn(0x7fdae80d0e10 0x7fdae80cd3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:45280/0 (socket says 172.21.15.44:45280) 2024-04-22T07:04:42.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaedfdf640 1 -- 172.21.15.44:0/2914741792 >> v2:172.21.15.44:3300/0 conn(0x7fdae80cbcc0 msgr2=0x7fdae80d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:42.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaedfdf640 1 --2- 172.21.15.44:0/2914741792 >> v2:172.21.15.44:3300/0 conn(0x7fdae80cbcc0 0x7fdae80d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaedfdf640 1 -- 172.21.15.44:0/2914741792 >> v1:172.21.15.70:6789/0 conn(0x7fdae80d0e10 legacy=0x7fdae80cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaecfdd640 1 -- 172.21.15.44:0/2914741792 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709886897 0 0) 0x7fdae815af90 con 0x7fdae80d0e10 2024-04-22T07:04:42.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.678+0000 7fdaedfdf640 1 -- 172.21.15.44:0/2914741792 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae815b550 con 0x7fdae8155040 2024-04-22T07:04:42.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.679+0000 7fdaedfdf640 1 --2- 172.21.15.44:0/2914741792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 0x7fdae8072c10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009870 tx=0x7fdae002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c98f0ce376f58380 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.679+0000 7fdaecfdd640 1 -- 172.21.15.44:0/2914741792 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0034070 con 0x7fdae8155040 2024-04-22T07:04:42.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.679+0000 7fdaecfdd640 1 -- 172.21.15.44:0/2914741792 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae0030840 con 0x7fdae8155040 2024-04-22T07:04:42.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.679+0000 7fdaecfdd640 1 -- 172.21.15.44:0/2914741792 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0030c10 con 0x7fdae8155040 2024-04-22T07:04:42.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.679+0000 7fdaf0243640 1 -- 172.21.15.44:0/2914741792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 msgr2=0x7fdae8072c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.679+0000 7fdaf0243640 1 --2- 172.21.15.44:0/2914741792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 0x7fdae8072c10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009870 tx=0x7fdae002b980 comp rx=0 tx=0).stop 2024-04-22T07:04:42.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.680+0000 7fdaf0243640 1 -- 172.21.15.44:0/2914741792 shutdown_connections 2024-04-22T07:04:42.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.680+0000 7fdaf0243640 1 --2- 172.21.15.44:0/2914741792 >> v2:172.21.15.44:3300/0 conn(0x7fdae80cbcc0 0x7fdae80d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.680+0000 7fdaf0243640 1 --2- 172.21.15.44:0/2914741792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 0x7fdae8072c10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.680+0000 7fdaf0243640 1 -- 172.21.15.44:0/2914741792 >> 172.21.15.44:0/2914741792 conn(0x7fdae806caa0 msgr2=0x7fdae806ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.680+0000 7fdaf0243640 1 -- 172.21.15.44:0/2914741792 shutdown_connections 2024-04-22T07:04:42.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.680+0000 7fdaf0243640 1 -- 172.21.15.44:0/2914741792 wait complete. 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.681+0000 7fdaf0243640 1 Processor -- start 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.681+0000 7fdaf0243640 1 -- start start 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.681+0000 7fdaf0243640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdae80cbcc0 0x7fdae80b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.681+0000 7fdaf0243640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdae80d0e10 0x7fdae80cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaf0243640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 0x7fdae80d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaf0243640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdae80cceb0 con 0x7fdae80cbcc0 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaf0243640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdae815bb10 con 0x7fdae80d0e10 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaf0243640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdae815ac80 con 0x7fdae8155040 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaedfdf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdae80cbcc0 0x7fdae80b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaedfdf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdae80cbcc0 0x7fdae80b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41204/0 (socket says 172.21.15.44:41204) 2024-04-22T07:04:42.681 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaedfdf640 1 -- 172.21.15.44:0/3420238275 learned_addr learned my addr 172.21.15.44:0/3420238275 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaed7de640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdae80d0e10 0x7fdae80cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaee7e0640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 0x7fdae80d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaedfdf640 1 -- 172.21.15.44:0/3420238275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 msgr2=0x7fdae80d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaedfdf640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 0x7fdae80d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaedfdf640 1 -- 172.21.15.44:0/3420238275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdae80d0e10 msgr2=0x7fdae80cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaedfdf640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdae80d0e10 0x7fdae80cfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaedfdf640 1 -- 172.21.15.44:0/3420238275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae80b55a0 con 0x7fdae80cbcc0 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaedfdf640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdae80cbcc0 0x7fdae80b3d30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdae00090b0 tx=0x7fdae00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdae67fc640 1 -- 172.21.15.44:0/3420238275 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0034040 con 0x7fdae80cbcc0 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdae67fc640 1 -- 172.21.15.44:0/3420238275 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae0030640 con 0x7fdae80cbcc0 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaf0243640 1 -- 172.21.15.44:0/3420238275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae80b5880 con 0x7fdae80cbcc0 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdae67fc640 1 -- 172.21.15.44:0/3420238275 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0039d40 con 0x7fdae80cbcc0 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.682+0000 7fdaf0243640 1 -- 172.21.15.44:0/3420238275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae814a600 con 0x7fdae80cbcc0 2024-04-22T07:04:42.682 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.683+0000 7fdaf0243640 1 -- 172.21.15.44:0/3420238275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae814a9e0 con 0x7fdae80cbcc0 2024-04-22T07:04:42.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.684+0000 7fdae67fc640 1 -- 172.21.15.44:0/3420238275 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdae0037070 con 0x7fdae80cbcc0 2024-04-22T07:04:42.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.684+0000 7fdae67fc640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdad406c8f0 0x7fdad406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.684+0000 7fdae67fc640 1 -- 172.21.15.44:0/3420238275 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdae00ad7b0 con 0x7fdae80cbcc0 2024-04-22T07:04:42.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.684+0000 7fdaed7de640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdad406c8f0 0x7fdad406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.683 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.685+0000 7fdaed7de640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdad406c8f0 0x7fdad406edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdae8158820 tx=0x7fdadc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.689+0000 7fdae67fc640 1 -- 172.21.15.44:0/3420238275 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdae007b6a0 con 0x7fdae80cbcc0 2024-04-22T07:04:42.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.873+0000 7fdaf0243640 1 -- 172.21.15.44:0/3420238275 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdae8068b00 con 0x7fdae80cbcc0 2024-04-22T07:04:42.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.874+0000 7fdae67fc640 1 -- 172.21.15.44:0/3420238275 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fdae007f9a0 con 0x7fdae80cbcc0 2024-04-22T07:04:42.872 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T07:04:42.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.875+0000 7fdaf0243640 1 -- 172.21.15.44:0/3420238275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdad406c8f0 msgr2=0x7fdad406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.875+0000 7fdaf0243640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdad406c8f0 0x7fdad406edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdae8158820 tx=0x7fdadc009190 comp rx=0 tx=0).stop 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.875+0000 7fdaf0243640 1 -- 172.21.15.44:0/3420238275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdae80cbcc0 msgr2=0x7fdae80b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.875+0000 7fdaf0243640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdae80cbcc0 0x7fdae80b3d30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdae00090b0 tx=0x7fdae00046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.875+0000 7fdaf0243640 1 -- 172.21.15.44:0/3420238275 shutdown_connections 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.875+0000 7fdaf0243640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdad406c8f0 0x7fdad406edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.875+0000 7fdaf0243640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdae8155040 0x7fdae80d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.875+0000 7fdaf0243640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdae80d0e10 0x7fdae80cfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.875+0000 7fdaf0243640 1 --2- 172.21.15.44:0/3420238275 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdae80cbcc0 0x7fdae80b3d30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.875+0000 7fdaf0243640 1 -- 172.21.15.44:0/3420238275 >> 172.21.15.44:0/3420238275 conn(0x7fdae806caa0 msgr2=0x7fdae806ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.876+0000 7fdaf0243640 1 -- 172.21.15.44:0/3420238275 shutdown_connections 2024-04-22T07:04:42.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.876+0000 7fdaf0243640 1 -- 172.21.15.44:0/3420238275 wait complete. 2024-04-22T07:04:42.884 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:04:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.990+0000 7fedc25f4640 1 Processor -- start 2024-04-22T07:04:42.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.990+0000 7fedc25f4640 1 -- start start 2024-04-22T07:04:42.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc25f4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fedbc1552c0 0x7fedbc0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc25f4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fedbc0cc920 0x7fedbc0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc25f4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedbc1556a0 con 0x7fedbc0caec0 2024-04-22T07:04:42.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc25f4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fedbc15b160 con 0x7fedbc0cc920 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc25f4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fedbc15b440 con 0x7fedbc1552c0 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc0b91640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fedbc0cc920 0x7fedbc0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc0b91640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fedbc0cc920 0x7fedbc0cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41210/0 (socket says 172.21.15.44:41210) 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc0b91640 1 -- 172.21.15.44:0/3084138782 learned_addr learned my addr 172.21.15.44:0/3084138782 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedbbfff640 1 --2- 172.21.15.44:0/3084138782 >> v2:172.21.15.70:3300/0 conn(0x7fedbc1552c0 0x7fedbc0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc0b91640 1 -- 172.21.15.44:0/3084138782 >> v2:172.21.15.70:3300/0 conn(0x7fedbc1552c0 msgr2=0x7fedbc0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc0b91640 1 --2- 172.21.15.44:0/3084138782 >> v2:172.21.15.70:3300/0 conn(0x7fedbc1552c0 0x7fedbc0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc0b91640 1 -- 172.21.15.44:0/3084138782 >> v1:172.21.15.70:6789/0 conn(0x7fedbc0caec0 legacy=0x7fedbc0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedc0b91640 1 -- 172.21.15.44:0/3084138782 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc15b720 con 0x7fedbc0cc920 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedbaffd640 1 -- 172.21.15.44:0/3084138782 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92232854 0 0) 0x7fedbc1556a0 con 0x7fedbc0caec0 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.991+0000 7fedbbfff640 1 --2- 172.21.15.44:0/3084138782 >> v2:172.21.15.70:3300/0 conn(0x7fedbc1552c0 0x7fedbc0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:42.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.992+0000 7fedc0b91640 1 --2- 172.21.15.44:0/3084138782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 0x7fedbc0cd310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0d09f0 tx=0x7fedb402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730ae1ff52ccd09b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.992+0000 7fedbaffd640 1 -- 172.21.15.44:0/3084138782 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb402cb20 con 0x7fedbc0cc920 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.992+0000 7fedbaffd640 1 -- 172.21.15.44:0/3084138782 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb402ccc0 con 0x7fedbc0cc920 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.992+0000 7fedbaffd640 1 -- 172.21.15.44:0/3084138782 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb4034610 con 0x7fedbc0cc920 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.993+0000 7fedc25f4640 1 -- 172.21.15.44:0/3084138782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 msgr2=0x7fedbc0cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.993+0000 7fedc25f4640 1 --2- 172.21.15.44:0/3084138782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 0x7fedbc0cd310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0d09f0 tx=0x7fedb402bf80 comp rx=0 tx=0).stop 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.993+0000 7fedc25f4640 1 -- 172.21.15.44:0/3084138782 shutdown_connections 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.993+0000 7fedc25f4640 1 --2- 172.21.15.44:0/3084138782 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 0x7fedbc0cd310 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.993+0000 7fedc25f4640 1 --2- 172.21.15.44:0/3084138782 >> v2:172.21.15.70:3300/0 conn(0x7fedbc1552c0 0x7fedbc0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.993+0000 7fedc25f4640 1 -- 172.21.15.44:0/3084138782 >> 172.21.15.44:0/3084138782 conn(0x7fedbc0676b0 msgr2=0x7fedbc067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.993+0000 7fedc25f4640 1 -- 172.21.15.44:0/3084138782 shutdown_connections 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.993+0000 7fedc25f4640 1 -- 172.21.15.44:0/3084138782 wait complete. 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.994+0000 7fedc25f4640 1 Processor -- start 2024-04-22T07:04:42.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.994+0000 7fedc25f4640 1 -- start start 2024-04-22T07:04:42.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedc25f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fedbc0caec0 0x7fedbc15c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedc25f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 0x7fedbc15dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedc25f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedbc1552c0 0x7fedbc1f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedc25f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc0c6a00 con 0x7fedbc0cc920 2024-04-22T07:04:42.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedc25f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc15bce0 con 0x7fedbc1552c0 2024-04-22T07:04:42.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedc25f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fedbc0d1130 con 0x7fedbc0caec0 2024-04-22T07:04:42.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedbb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 0x7fedbc15dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedbb7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 0x7fedbc15dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41226/0 (socket says 172.21.15.44:41226) 2024-04-22T07:04:42.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedbb7fe640 1 -- 172.21.15.44:0/1049760521 learned_addr learned my addr 172.21.15.44:0/1049760521 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:42.995 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedc0b91640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedbc1552c0 0x7fedbc1f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedbbfff640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fedbc0caec0 0x7fedbc15c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedbb7fe640 1 -- 172.21.15.44:0/1049760521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fedbc0caec0 msgr2=0x7fedbc15c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedbb7fe640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fedbc0caec0 0x7fedbc15c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedbb7fe640 1 -- 172.21.15.44:0/1049760521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedbc1552c0 msgr2=0x7fedbc1f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedbb7fe640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedbc1552c0 0x7fedbc1f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedbb7fe640 1 -- 172.21.15.44:0/1049760521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc15c610 con 0x7fedbc0cc920 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedbb7fe640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 0x7fedbc15dbd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedac00c520 tx=0x7fedac00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedb8ff9640 1 -- 172.21.15.44:0/1049760521 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac017070 con 0x7fedbc0cc920 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedb8ff9640 1 -- 172.21.15.44:0/1049760521 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedac0052d0 con 0x7fedbc0cc920 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedc25f4640 1 -- 172.21.15.44:0/1049760521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedbc0b3070 con 0x7fedbc0cc920 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedb8ff9640 1 -- 172.21.15.44:0/1049760521 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac012870 con 0x7fedbc0cc920 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.995+0000 7fedc25f4640 1 -- 172.21.15.44:0/1049760521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedbc1f0890 con 0x7fedbc0cc920 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.996+0000 7fedc25f4640 1 -- 172.21.15.44:0/1049760521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedbc068b00 con 0x7fedbc0cc920 2024-04-22T07:04:42.996 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.997+0000 7fedb8ff9640 1 -- 172.21.15.44:0/1049760521 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedac01a070 con 0x7fedbc0cc920 2024-04-22T07:04:42.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.997+0000 7fedb8ff9640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feda806c8f0 0x7feda806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:42.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.997+0000 7fedb8ff9640 1 -- 172.21.15.44:0/1049760521 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fedac08ade0 con 0x7fedbc0cc920 2024-04-22T07:04:42.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.997+0000 7fedbbfff640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feda806c8f0 0x7feda806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:42.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:42.998+0000 7fedbbfff640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feda806c8f0 0x7feda806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009560 tx=0x7fedb0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:42.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.002+0000 7fedb8ff9640 1 -- 172.21.15.44:0/1049760521 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedac058bd0 con 0x7fedbc0cc920 2024-04-22T07:04:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.187+0000 7fedc25f4640 1 -- 172.21.15.44:0/1049760521 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fedbc0cf5b0 con 0x7fedbc0cc920 2024-04-22T07:04:43.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.187+0000 7fedb8ff9640 1 -- 172.21.15.44:0/1049760521 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fedac05ced0 con 0x7fedbc0cc920 2024-04-22T07:04:43.185 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T07:04:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 -- 172.21.15.44:0/1049760521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feda806c8f0 msgr2=0x7feda806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feda806c8f0 0x7feda806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009560 tx=0x7fedb0009190 comp rx=0 tx=0).stop 2024-04-22T07:04:43.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 -- 172.21.15.44:0/1049760521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 msgr2=0x7fedbc15dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 0x7fedbc15dbd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fedac00c520 tx=0x7fedac00c9e0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 -- 172.21.15.44:0/1049760521 shutdown_connections 2024-04-22T07:04:43.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7feda806c8f0 0x7feda806edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedbc1552c0 0x7fedbc1f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fedbc0cc920 0x7fedbc15dbd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 --2- 172.21.15.44:0/1049760521 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fedbc0caec0 0x7fedbc15c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 -- 172.21.15.44:0/1049760521 >> 172.21.15.44:0/1049760521 conn(0x7fedbc0676b0 msgr2=0x7fedbc0cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 -- 172.21.15.44:0/1049760521 shutdown_connections 2024-04-22T07:04:43.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.189+0000 7fedc25f4640 1 -- 172.21.15.44:0/1049760521 wait complete. 2024-04-22T07:04:43.198 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:43.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.300+0000 7f9c3c4e4640 1 Processor -- start 2024-04-22T07:04:43.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.300+0000 7f9c3c4e4640 1 -- start start 2024-04-22T07:04:43.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3c4e4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 0x7f9c3415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3c4e4640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c3415ee90 con 0x7f9c340cb3e0 2024-04-22T07:04:43.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3c4e4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c3415f170 con 0x7f9c340cd6f0 2024-04-22T07:04:43.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3c4e4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9c340cd260 con 0x7f9c34154f70 2024-04-22T07:04:43.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3aa81640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9c340cb3e0 0x7f9c3415e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55682/0 (socket says 172.21.15.44:55682) 2024-04-22T07:04:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3aa81640 1 -- 172.21.15.44:0/1812122913 learned_addr learned my addr 172.21.15.44:0/1812122913 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3a280640 1 --2- 172.21.15.44:0/1812122913 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 0x7f9c3415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3927e640 1 -- 172.21.15.44:0/1812122913 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965693301 0 0) 0x7f9c3415ee90 con 0x7f9c340cb3e0 2024-04-22T07:04:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3927e640 1 -- 172.21.15.44:0/1812122913 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c1c003520 con 0x7f9c340cb3e0 2024-04-22T07:04:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3927e640 1 -- 172.21.15.44:0/1812122913 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884122740 0 0) 0x7f9c3415f170 con 0x7f9c340cd6f0 2024-04-22T07:04:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3927e640 1 -- 172.21.15.44:0/1812122913 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c3415ee90 con 0x7f9c340cd6f0 2024-04-22T07:04:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3a280640 1 -- 172.21.15.44:0/1812122913 >> v1:172.21.15.70:6789/0 conn(0x7f9c340cd6f0 legacy=0x7f9c340c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3a280640 1 -- 172.21.15.44:0/1812122913 >> v1:172.21.15.44:6789/0 conn(0x7f9c340cb3e0 legacy=0x7f9c3415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.301+0000 7f9c3a280640 1 -- 172.21.15.44:0/1812122913 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c340cc930 con 0x7f9c34154f70 2024-04-22T07:04:43.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.302+0000 7f9c3a280640 1 --2- 172.21.15.44:0/1812122913 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 0x7f9c3415ad30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0097d0 tx=0x7f9c2c02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda53bda5f67b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.302+0000 7f9c3927e640 1 -- 172.21.15.44:0/1812122913 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c034070 con 0x7f9c34154f70 2024-04-22T07:04:43.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.302+0000 7f9c3927e640 1 -- 172.21.15.44:0/1812122913 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2c030880 con 0x7f9c34154f70 2024-04-22T07:04:43.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.302+0000 7f9c3927e640 1 -- 172.21.15.44:0/1812122913 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c030c30 con 0x7f9c34154f70 2024-04-22T07:04:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.303+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1812122913 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 msgr2=0x7f9c3415ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.303+0000 7f9c3c4e4640 1 --2- 172.21.15.44:0/1812122913 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 0x7f9c3415ad30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0097d0 tx=0x7f9c2c02b9c0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.303+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1812122913 shutdown_connections 2024-04-22T07:04:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.303+0000 7f9c3c4e4640 1 --2- 172.21.15.44:0/1812122913 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 0x7f9c3415ad30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.303+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1812122913 >> 172.21.15.44:0/1812122913 conn(0x7f9c34067d40 msgr2=0x7f9c3406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.303+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1812122913 shutdown_connections 2024-04-22T07:04:43.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.303+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1812122913 wait complete. 2024-04-22T07:04:43.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.304+0000 7f9c3c4e4640 1 Processor -- start 2024-04-22T07:04:43.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.304+0000 7f9c3c4e4640 1 -- start start 2024-04-22T07:04:43.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c3c4e4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c340cb3e0 0x7f9c34155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c3c4e4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c340cd6f0 0x7f9c3415d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c3c4e4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 0x7f9c341e7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c3c4e4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c34155350 con 0x7f9c340cd6f0 2024-04-22T07:04:43.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c3c4e4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9c340ccef0 con 0x7f9c340cb3e0 2024-04-22T07:04:43.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c3c4e4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9c341f1ce0 con 0x7f9c34154f70 2024-04-22T07:04:43.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c39a7f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c340cd6f0 0x7f9c3415d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c3aa81640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 0x7f9c341e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c3a280640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c340cb3e0 0x7f9c34155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c39a7f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c340cd6f0 0x7f9c3415d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41242/0 (socket says 172.21.15.44:41242) 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c39a7f640 1 -- 172.21.15.44:0/1672378330 learned_addr learned my addr 172.21.15.44:0/1672378330 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c39a7f640 1 -- 172.21.15.44:0/1672378330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 msgr2=0x7f9c341e7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c39a7f640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 0x7f9c341e7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c39a7f640 1 -- 172.21.15.44:0/1672378330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c340cb3e0 msgr2=0x7f9c34155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c39a7f640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c340cb3e0 0x7f9c34155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c39a7f640 1 -- 172.21.15.44:0/1672378330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c341f51d0 con 0x7f9c340cd6f0 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c3a280640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c340cb3e0 0x7f9c34155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.305+0000 7f9c39a7f640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c340cd6f0 0x7f9c3415d8c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800c530 tx=0x7f9c2800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.306+0000 7f9c32ffd640 1 -- 172.21.15.44:0/1672378330 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c280052d0 con 0x7f9c340cd6f0 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.306+0000 7f9c32ffd640 1 -- 172.21.15.44:0/1672378330 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c28005490 con 0x7f9c340cd6f0 2024-04-22T07:04:43.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.306+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1672378330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c341f0ec0 con 0x7f9c340cd6f0 2024-04-22T07:04:43.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.306+0000 7f9c32ffd640 1 -- 172.21.15.44:0/1672378330 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28012870 con 0x7f9c340cd6f0 2024-04-22T07:04:43.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.306+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1672378330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c341f1720 con 0x7f9c340cd6f0 2024-04-22T07:04:43.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.307+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1672378330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c341f11c0 con 0x7f9c340cd6f0 2024-04-22T07:04:43.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.307+0000 7f9c32ffd640 1 -- 172.21.15.44:0/1672378330 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c28012a10 con 0x7f9c340cd6f0 2024-04-22T07:04:43.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.308+0000 7f9c32ffd640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c2006c8f0 0x7f9c2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.306 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.308+0000 7f9c3a280640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c2006c8f0 0x7f9c2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.308+0000 7f9c32ffd640 1 -- 172.21.15.44:0/1672378330 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9c2808b6c0 con 0x7f9c340cd6f0 2024-04-22T07:04:43.307 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.308+0000 7f9c3a280640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c2006c8f0 0x7f9c2006edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c036000 tx=0x7f9c2c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.309 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.312+0000 7f9c32ffd640 1 -- 172.21.15.44:0/1672378330 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c341f11c0 con 0x7f9c340cd6f0 2024-04-22T07:04:43.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.534+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1672378330 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c340cc500 con 0x7f9c340cd6f0 2024-04-22T07:04:43.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.534+0000 7f9c32ffd640 1 -- 172.21.15.44:0/1672378330 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6904 (secure 0 0 0) 0x7f9c280594b0 con 0x7f9c340cd6f0 2024-04-22T07:04:43.532 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:43.532 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:04:39.933702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4324,"mds_1":4303,"mds_2":4300,"mds_3":4278,"mds_4":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"b","rank":3,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4324,"qdb_cluster":[4324,4303,4300,4278]},"id":1}]} 2024-04-22T07:04:43.533 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 16 2024-04-22T07:04:43.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1672378330 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c2006c8f0 msgr2=0x7f9c2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c2006c8f0 0x7f9c2006edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c036000 tx=0x7f9c2c03c040 comp rx=0 tx=0).stop 2024-04-22T07:04:43.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1672378330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c340cd6f0 msgr2=0x7f9c3415d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c340cd6f0 0x7f9c3415d8c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800c530 tx=0x7f9c2800c9f0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.534 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1672378330 shutdown_connections 2024-04-22T07:04:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c2006c8f0 0x7f9c2006edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c34154f70 0x7f9c341e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c340cd6f0 0x7f9c3415d8c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 --2- 172.21.15.44:0/1672378330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c340cb3e0 0x7f9c34155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1672378330 >> 172.21.15.44:0/1672378330 conn(0x7f9c34067d40 msgr2=0x7f9c34071f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1672378330 shutdown_connections 2024-04-22T07:04:43.535 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.536+0000 7f9c3c4e4640 1 -- 172.21.15.44:0/1672378330 wait complete. 2024-04-22T07:04:43.545 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.647+0000 7f9c8a6d2640 1 Processor -- start 2024-04-22T07:04:43.645 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.647+0000 7f9c8a6d2640 1 -- start start 2024-04-22T07:04:43.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c8a6d2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 0x7f9c8415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c8a6d2640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c8415ee90 con 0x7f9c840cd6f0 2024-04-22T07:04:43.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c8a6d2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c8415f170 con 0x7f9c840cb3e0 2024-04-22T07:04:43.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c8a6d2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9c840cd260 con 0x7f9c84154f70 2024-04-22T07:04:43.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c837fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f9c840cd6f0 0x7f9c840c5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:55684/0 (socket says 172.21.15.44:55684) 2024-04-22T07:04:43.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c837fe640 1 -- 172.21.15.44:0/1298779911 learned_addr learned my addr 172.21.15.44:0/1298779911 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:43.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c83fff640 1 --2- 172.21.15.44:0/1298779911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 0x7f9c8415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c82ffd640 1 -- 172.21.15.44:0/1298779911 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 487674601 0 0) 0x7f9c8415ee90 con 0x7f9c840cd6f0 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c82ffd640 1 -- 172.21.15.44:0/1298779911 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c70003520 con 0x7f9c840cd6f0 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c82ffd640 1 -- 172.21.15.44:0/1298779911 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3565910452 0 0) 0x7f9c8415f170 con 0x7f9c840cb3e0 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c82ffd640 1 -- 172.21.15.44:0/1298779911 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c8415ee90 con 0x7f9c840cb3e0 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c83fff640 1 -- 172.21.15.44:0/1298779911 >> v1:172.21.15.70:6789/0 conn(0x7f9c840cb3e0 legacy=0x7f9c8415e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c83fff640 1 -- 172.21.15.44:0/1298779911 >> v1:172.21.15.44:6789/0 conn(0x7f9c840cd6f0 legacy=0x7f9c840c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.648+0000 7f9c83fff640 1 -- 172.21.15.44:0/1298779911 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c840cc930 con 0x7f9c84154f70 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.649+0000 7f9c83fff640 1 --2- 172.21.15.44:0/1298779911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 0x7f9c8415ad30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c0097d0 tx=0x7f9c6c02b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5cd5f2925134b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.649+0000 7f9c82ffd640 1 -- 172.21.15.44:0/1298779911 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c034070 con 0x7f9c84154f70 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.649+0000 7f9c82ffd640 1 -- 172.21.15.44:0/1298779911 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6c030880 con 0x7f9c84154f70 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.649+0000 7f9c82ffd640 1 -- 172.21.15.44:0/1298779911 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c030c50 con 0x7f9c84154f70 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.650+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/1298779911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 msgr2=0x7f9c8415ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.650+0000 7f9c8a6d2640 1 --2- 172.21.15.44:0/1298779911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 0x7f9c8415ad30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c0097d0 tx=0x7f9c6c02b9c0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.650+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/1298779911 shutdown_connections 2024-04-22T07:04:43.652 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.650+0000 7f9c8a6d2640 1 --2- 172.21.15.44:0/1298779911 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 0x7f9c8415ad30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.650+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/1298779911 >> 172.21.15.44:0/1298779911 conn(0x7f9c84067d40 msgr2=0x7f9c8406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.650+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/1298779911 shutdown_connections 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.650+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/1298779911 wait complete. 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.651+0000 7f9c8a6d2640 1 Processor -- start 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.651+0000 7f9c8a6d2640 1 -- start start 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c8a6d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c840cb3e0 0x7f9c8415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c8a6d2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c840cd6f0 0x7f9c8414b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c8a6d2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 0x7f9c841f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c8a6d2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84155350 con 0x7f9c840cd6f0 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c8a6d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9c840ccef0 con 0x7f9c840cb3e0 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c8a6d2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8414c2f0 con 0x7f9c84154f70 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c837fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c840cd6f0 0x7f9c8414b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c837fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c840cd6f0 0x7f9c8414b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41254/0 (socket says 172.21.15.44:41254) 2024-04-22T07:04:43.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c837fe640 1 -- 172.21.15.44:0/2767510270 learned_addr learned my addr 172.21.15.44:0/2767510270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c88c6f640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 0x7f9c841f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c83fff640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c840cb3e0 0x7f9c8415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c837fe640 1 -- 172.21.15.44:0/2767510270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 msgr2=0x7f9c841f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c837fe640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 0x7f9c841f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c837fe640 1 -- 172.21.15.44:0/2767510270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c840cb3e0 msgr2=0x7f9c8415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c837fe640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c840cb3e0 0x7f9c8415d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c837fe640 1 -- 172.21.15.44:0/2767510270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8415e0b0 con 0x7f9c840cd6f0 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.652+0000 7f9c83fff640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c840cb3e0 0x7f9c8415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.653+0000 7f9c837fe640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c840cd6f0 0x7f9c8414b1b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400c500 tx=0x7f9c7400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.653+0000 7f9c80ff9640 1 -- 172.21.15.44:0/2767510270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74017070 con 0x7f9c840cd6f0 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.653+0000 7f9c80ff9640 1 -- 172.21.15.44:0/2767510270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c74005d00 con 0x7f9c840cd6f0 2024-04-22T07:04:43.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.653+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/2767510270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8414bd30 con 0x7f9c840cd6f0 2024-04-22T07:04:43.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.653+0000 7f9c80ff9640 1 -- 172.21.15.44:0/2767510270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74007330 con 0x7f9c840cd6f0 2024-04-22T07:04:43.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.653+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/2767510270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c841f4430 con 0x7f9c840cd6f0 2024-04-22T07:04:43.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.654+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/2767510270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c841f4630 con 0x7f9c840cd6f0 2024-04-22T07:04:43.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.654+0000 7f9c80ff9640 1 -- 172.21.15.44:0/2767510270 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c7401a070 con 0x7f9c840cd6f0 2024-04-22T07:04:43.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.655+0000 7f9c80ff9640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c6406c8f0 0x7f9c6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.655+0000 7f9c80ff9640 1 -- 172.21.15.44:0/2767510270 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9c7408af00 con 0x7f9c840cd6f0 2024-04-22T07:04:43.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.655+0000 7f9c83fff640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c6406c8f0 0x7f9c6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.655+0000 7f9c83fff640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c6406c8f0 0x7f9c6406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c036000 tx=0x7f9c6c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.659+0000 7f9c80ff9640 1 -- 172.21.15.44:0/2767510270 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c841f4630 con 0x7f9c840cd6f0 2024-04-22T07:04:43.844 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.846+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/2767510270 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c840cc500 con 0x7f9c840cd6f0 2024-04-22T07:04:43.845 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.847+0000 7f9c80ff9640 1 -- 172.21.15.44:0/2767510270 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16557 (secure 0 0 0) 0x7f9c74058da0 con 0x7f9c840cd6f0 2024-04-22T07:04:43.845 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:43.846 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":23,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:14.778262+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:43.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/2767510270 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c6406c8f0 msgr2=0x7f9c6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c6406c8f0 0x7f9c6406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c036000 tx=0x7f9c6c03b040 comp rx=0 tx=0).stop 2024-04-22T07:04:43.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/2767510270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c840cd6f0 msgr2=0x7f9c8414b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.847 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c840cd6f0 0x7f9c8414b1b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400c500 tx=0x7f9c7400c9c0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/2767510270 shutdown_connections 2024-04-22T07:04:43.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9c6406c8f0 0x7f9c6406edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9c84154f70 0x7f9c841f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9c840cd6f0 0x7f9c8414b1b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 --2- 172.21.15.44:0/2767510270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c840cb3e0 0x7f9c8415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/2767510270 >> 172.21.15.44:0/2767510270 conn(0x7f9c84067d40 msgr2=0x7f9c840c6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/2767510270 shutdown_connections 2024-04-22T07:04:43.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.849+0000 7f9c8a6d2640 1 -- 172.21.15.44:0/2767510270 wait complete. 2024-04-22T07:04:43.858 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.962+0000 7f46add04640 1 Processor -- start 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.962+0000 7f46add04640 1 -- start start 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46add04640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cd6f0 0x7f46a815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46add04640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f46a80cb3e0 0x7f46a80cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46add04640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a815ee90 con 0x7f46a8154f70 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46add04640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f46a80cd260 con 0x7f46a80cb3e0 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46add04640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f46a815f170 con 0x7f46a80cd6f0 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46a7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f46a80cb3e0 0x7f46a80cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46a6ffd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cd6f0 0x7f46a815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46a77fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f46a8154f70 0x7f46a80c5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:45306/0 (socket says 172.21.15.44:45306) 2024-04-22T07:04:43.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46a77fe640 1 -- 172.21.15.44:0/1655898549 learned_addr learned my addr 172.21.15.44:0/1655898549 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46a7fff640 1 -- 172.21.15.44:0/1655898549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cd6f0 msgr2=0x7f46a815ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46a7fff640 1 --2- 172.21.15.44:0/1655898549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cd6f0 0x7f46a815ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46a7fff640 1 -- 172.21.15.44:0/1655898549 >> v1:172.21.15.70:6789/0 conn(0x7f46a8154f70 legacy=0x7f46a80c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46a7fff640 1 -- 172.21.15.44:0/1655898549 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a80cc930 con 0x7f46a80cb3e0 2024-04-22T07:04:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46a67fc640 1 -- 172.21.15.44:0/1655898549 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945883911 0 0) 0x7f46a815ee90 con 0x7f46a8154f70 2024-04-22T07:04:43.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.963+0000 7f46a6ffd640 1 --2- 172.21.15.44:0/1655898549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cd6f0 0x7f46a815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.964+0000 7f46a7fff640 1 --2- 172.21.15.44:0/1655898549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a80cb3e0 0x7f46a80cb890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009d90 tx=0x7f46a002bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d44e81b30b138101 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.964+0000 7f46a67fc640 1 -- 172.21.15.44:0/1655898549 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0039070 con 0x7f46a80cb3e0 2024-04-22T07:04:43.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.964+0000 7f46a67fc640 1 -- 172.21.15.44:0/1655898549 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a002cce0 con 0x7f46a80cb3e0 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.964+0000 7f46a67fc640 1 -- 172.21.15.44:0/1655898549 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0034660 con 0x7f46a80cb3e0 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.964+0000 7f46add04640 1 -- 172.21.15.44:0/1655898549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a80cb3e0 msgr2=0x7f46a80cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.964+0000 7f46add04640 1 --2- 172.21.15.44:0/1655898549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a80cb3e0 0x7f46a80cb890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009d90 tx=0x7f46a002bf30 comp rx=0 tx=0).stop 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.965+0000 7f46add04640 1 -- 172.21.15.44:0/1655898549 shutdown_connections 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.965+0000 7f46add04640 1 --2- 172.21.15.44:0/1655898549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a80cb3e0 0x7f46a80cb890 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.965+0000 7f46add04640 1 --2- 172.21.15.44:0/1655898549 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cd6f0 0x7f46a815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.965+0000 7f46add04640 1 -- 172.21.15.44:0/1655898549 >> 172.21.15.44:0/1655898549 conn(0x7f46a8067d40 msgr2=0x7f46a806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.965+0000 7f46add04640 1 -- 172.21.15.44:0/1655898549 shutdown_connections 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.965+0000 7f46add04640 1 -- 172.21.15.44:0/1655898549 wait complete. 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.966+0000 7f46add04640 1 Processor -- start 2024-04-22T07:04:43.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.966+0000 7f46add04640 1 -- start start 2024-04-22T07:04:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.966+0000 7f46add04640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cb3e0 0x7f46a8158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.966+0000 7f46add04640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a80cd6f0 0x7f46a8155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.966+0000 7f46add04640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a8154f70 0x7f46a81f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46add04640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8155350 con 0x7f46a8154f70 2024-04-22T07:04:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46add04640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f46a80ccef0 con 0x7f46a80cd6f0 2024-04-22T07:04:43.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46add04640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f46a8158d40 con 0x7f46a80cb3e0 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a8154f70 0x7f46a81f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a8154f70 0x7f46a81f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:41260/0 (socket says 172.21.15.44:41260) 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a7fff640 1 -- 172.21.15.44:0/800189022 learned_addr learned my addr 172.21.15.44:0/800189022 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a77fe640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cb3e0 0x7f46a8158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a6ffd640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a80cd6f0 0x7f46a8155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a77fe640 1 -- 172.21.15.44:0/800189022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a80cd6f0 msgr2=0x7f46a8155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a77fe640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a80cd6f0 0x7f46a8155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a77fe640 1 -- 172.21.15.44:0/800189022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a8154f70 msgr2=0x7f46a81f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a77fe640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a8154f70 0x7f46a81f4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a77fe640 1 -- 172.21.15.44:0/800189022 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a81f4cf0 con 0x7f46a80cb3e0 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a7fff640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a8154f70 0x7f46a81f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46a77fe640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cb3e0 0x7f46a8158250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f469800c410 tx=0x7f469800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f468ffff640 1 -- 172.21.15.44:0/800189022 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46980074e0 con 0x7f46a80cb3e0 2024-04-22T07:04:43.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f468ffff640 1 -- 172.21.15.44:0/800189022 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46980047e0 con 0x7f46a80cb3e0 2024-04-22T07:04:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46add04640 1 -- 172.21.15.44:0/800189022 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a81f4fd0 con 0x7f46a80cb3e0 2024-04-22T07:04:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f468ffff640 1 -- 172.21.15.44:0/800189022 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698004bd0 con 0x7f46a80cb3e0 2024-04-22T07:04:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.967+0000 7f46add04640 1 -- 172.21.15.44:0/800189022 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a8157a50 con 0x7f46a80cb3e0 2024-04-22T07:04:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.968+0000 7f46add04640 1 -- 172.21.15.44:0/800189022 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a80caba0 con 0x7f46a80cb3e0 2024-04-22T07:04:43.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.969+0000 7f468ffff640 1 -- 172.21.15.44:0/800189022 <== mon.2 v2:172.21.15.70:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4698011070 con 0x7f46a80cb3e0 2024-04-22T07:04:43.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.969+0000 7f468ffff640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f469406c8f0 0x7f469406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:43.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.969+0000 7f468ffff640 1 -- 172.21.15.44:0/800189022 <== mon.2 v2:172.21.15.70:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f469808b610 con 0x7f46a80cb3e0 2024-04-22T07:04:43.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.969+0000 7f46a6ffd640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f469406c8f0 0x7f469406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:43.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.970+0000 7f46a6ffd640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f469406c8f0 0x7f469406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f469c009560 tx=0x7f469c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:43.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:43.973+0000 7f468ffff640 1 -- 172.21.15.44:0/800189022 <== mon.2 v2:172.21.15.70:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4698059400 con 0x7f46a80cb3e0 2024-04-22T07:04:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.197+0000 7f46add04640 1 -- 172.21.15.44:0/800189022 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46a80cc500 con 0x7f46a80cb3e0 2024-04-22T07:04:44.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.198+0000 7f468ffff640 1 -- 172.21.15.44:0/800189022 <== mon.2 v2:172.21.15.70:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6904 (secure 0 0 0) 0x7f469805d700 con 0x7f46a80cb3e0 2024-04-22T07:04:44.196 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:44.196 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:04:39.933702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4324,"mds_1":4303,"mds_2":4300,"mds_3":4278,"mds_4":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"b","rank":3,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4324,"qdb_cluster":[4324,4303,4300,4278]},"id":1}]} 2024-04-22T07:04:44.196 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 16 2024-04-22T07:04:44.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 -- 172.21.15.44:0/800189022 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f469406c8f0 msgr2=0x7f469406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f469406c8f0 0x7f469406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f469c009560 tx=0x7f469c009190 comp rx=0 tx=0).stop 2024-04-22T07:04:44.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 -- 172.21.15.44:0/800189022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cb3e0 msgr2=0x7f46a8158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cb3e0 0x7f46a8158250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f469800c410 tx=0x7f469800c8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 -- 172.21.15.44:0/800189022 shutdown_connections 2024-04-22T07:04:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f469406c8f0 0x7f469406edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f46a8154f70 0x7f46a81f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46a80cd6f0 0x7f46a8155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 --2- 172.21.15.44:0/800189022 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f46a80cb3e0 0x7f46a8158250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 -- 172.21.15.44:0/800189022 >> 172.21.15.44:0/800189022 conn(0x7f46a8067d40 msgr2=0x7f46a80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 -- 172.21.15.44:0/800189022 shutdown_connections 2024-04-22T07:04:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:44.200+0000 7f46add04640 1 -- 172.21.15.44:0/800189022 wait complete. 2024-04-22T07:04:44.813 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-22T07:04:13.144503+0000', 'modified': '2024-04-22T07:04:39.933702+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4324, 'mds_1': 4303, 'mds_2': 4300, 'mds_3': 4278, 'mds_4': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'd', 'rank': 4, 'incarnation': 16, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.70:6834/1969018412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 1969018412}, {'type': 'v1', 'addr': '172.21.15.70:6834', 'nonce': 1969018412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4278': {'gid': 4278, 'name': 'b', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.70:6835/4058009549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6833', 'nonce': 4058009549}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 4058009549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4300': {'gid': 4300, 'name': 'c', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.44:6838/1520421043', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 1520421043}, {'type': 'v1', 'addr': '172.21.15.44:6838', 'nonce': 1520421043}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4303': {'gid': 4303, 'name': 'e', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.44:6839/1911557586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6837', 'nonce': 1911557586}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 1911557586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4324': {'gid': 4324, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.70:6837/2872131879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6836', 'nonce': 2872131879}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 2872131879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4324, 'qdb_cluster': [4324, 4303, 4300, 4278]} 2024-04-22T07:04:44.813 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4275:up:creating 2024-04-22T07:04:45.814 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.929+0000 7f4164243640 1 Processor -- start 2024-04-22T07:04:45.927 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.929+0000 7f4164243640 1 -- start start 2024-04-22T07:04:45.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f4164243640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 0x7f415c0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f4164243640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f415c15e8b0 0x7f415c15ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f4164243640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f415c155500 con 0x7f415c0caec0 2024-04-22T07:04:45.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f4164243640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f415c15f260 con 0x7f415c15e8b0 2024-04-22T07:04:45.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f4164243640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f415c072f90 con 0x7f415c155120 2024-04-22T07:04:45.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f41627e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f415c15e8b0 0x7f415c15ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f41627e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f415c15e8b0 0x7f415c15ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:40762/0 (socket says 172.21.15.44:40762) 2024-04-22T07:04:45.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f41617de640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f415c0caec0 0x7f415c0d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:54844/0 (socket says 172.21.15.44:54844) 2024-04-22T07:04:45.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f41627e0640 1 -- 172.21.15.44:0/2844133608 learned_addr learned my addr 172.21.15.44:0/2844133608 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:45.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f4161fdf640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 0x7f415c0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f41627e0640 1 -- 172.21.15.44:0/2844133608 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 msgr2=0x7f415c0c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:45.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f41627e0640 1 --2- 172.21.15.44:0/2844133608 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 0x7f415c0c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f41627e0640 1 -- 172.21.15.44:0/2844133608 >> v1:172.21.15.70:6789/0 conn(0x7f415c0caec0 legacy=0x7f415c0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f41627e0640 1 -- 172.21.15.44:0/2844133608 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c15b0d0 con 0x7f415c15e8b0 2024-04-22T07:04:45.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f4160fdd640 1 -- 172.21.15.44:0/2844133608 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2726310991 0 0) 0x7f415c155500 con 0x7f415c0caec0 2024-04-22T07:04:45.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.930+0000 7f4161fdf640 1 --2- 172.21.15.44:0/2844133608 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 0x7f415c0c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.931+0000 7f41627e0640 1 --2- 172.21.15.44:0/2844133608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f415c15e8b0 0x7f415c15ed20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4154009e50 tx=0x7f415402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad3fd387bdfb44 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.931+0000 7f4160fdd640 1 -- 172.21.15.44:0/2844133608 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415402caf0 con 0x7f415c15e8b0 2024-04-22T07:04:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.931+0000 7f4160fdd640 1 -- 172.21.15.44:0/2844133608 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f415402cc90 con 0x7f415c15e8b0 2024-04-22T07:04:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.931+0000 7f4160fdd640 1 -- 172.21.15.44:0/2844133608 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154034610 con 0x7f415c15e8b0 2024-04-22T07:04:45.930 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.932+0000 7f4164243640 1 -- 172.21.15.44:0/2844133608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f415c15e8b0 msgr2=0x7f415c15ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.932+0000 7f4164243640 1 --2- 172.21.15.44:0/2844133608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f415c15e8b0 0x7f415c15ed20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4154009e50 tx=0x7f415402bf50 comp rx=0 tx=0).stop 2024-04-22T07:04:45.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.932+0000 7f4164243640 1 -- 172.21.15.44:0/2844133608 shutdown_connections 2024-04-22T07:04:45.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.932+0000 7f4164243640 1 --2- 172.21.15.44:0/2844133608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f415c15e8b0 0x7f415c15ed20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.932+0000 7f4164243640 1 --2- 172.21.15.44:0/2844133608 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 0x7f415c0c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.932+0000 7f4164243640 1 -- 172.21.15.44:0/2844133608 >> 172.21.15.44:0/2844133608 conn(0x7f415c0676b0 msgr2=0x7f415c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:45.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.932+0000 7f4164243640 1 -- 172.21.15.44:0/2844133608 shutdown_connections 2024-04-22T07:04:45.931 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.932+0000 7f4164243640 1 -- 172.21.15.44:0/2844133608 wait complete. 2024-04-22T07:04:45.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.933+0000 7f4164243640 1 Processor -- start 2024-04-22T07:04:45.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.933+0000 7f4164243640 1 -- start start 2024-04-22T07:04:45.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4164243640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f415c0caec0 0x7f415c15a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4164243640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 0x7f415c14aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4164243640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f415c15e8b0 0x7f415c151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4164243640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f415c15b970 con 0x7f415c0caec0 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4164243640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f415c15b690 con 0x7f415c15e8b0 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4164243640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f415c15bc50 con 0x7f415c155120 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4161fdf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f415c0caec0 0x7f415c15a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f41617de640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 0x7f415c14aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4161fdf640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f415c0caec0 0x7f415c15a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46614/0 (socket says 172.21.15.44:46614) 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4161fdf640 1 -- 172.21.15.44:0/2520623681 learned_addr learned my addr 172.21.15.44:0/2520623681 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f41627e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f415c15e8b0 0x7f415c151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4161fdf640 1 -- 172.21.15.44:0/2520623681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 msgr2=0x7f415c14aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4161fdf640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 0x7f415c14aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4161fdf640 1 -- 172.21.15.44:0/2520623681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f415c15e8b0 msgr2=0x7f415c151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4161fdf640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f415c15e8b0 0x7f415c151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f4161fdf640 1 -- 172.21.15.44:0/2520623681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c159e20 con 0x7f415c0caec0 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.934+0000 7f41627e0640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f415c15e8b0 0x7f415c151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:45.934 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.935+0000 7f4161fdf640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f415c0caec0 0x7f415c15a520 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f415800ec10 tx=0x7f415800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.935+0000 7f41527fc640 1 -- 172.21.15.44:0/2520623681 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158017070 con 0x7f415c0caec0 2024-04-22T07:04:45.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.935+0000 7f41527fc640 1 -- 172.21.15.44:0/2520623681 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f415800cbd0 con 0x7f415c0caec0 2024-04-22T07:04:45.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.935+0000 7f4164243640 1 -- 172.21.15.44:0/2520623681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415c15ab80 con 0x7f415c0caec0 2024-04-22T07:04:45.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.935+0000 7f41527fc640 1 -- 172.21.15.44:0/2520623681 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158013470 con 0x7f415c0caec0 2024-04-22T07:04:45.935 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.935+0000 7f4164243640 1 -- 172.21.15.44:0/2520623681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415c1f4430 con 0x7f415c0caec0 2024-04-22T07:04:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.936+0000 7f4164243640 1 -- 172.21.15.44:0/2520623681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415c04fdc0 con 0x7f415c0caec0 2024-04-22T07:04:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.937+0000 7f41527fc640 1 -- 172.21.15.44:0/2520623681 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f415801a070 con 0x7f415c0caec0 2024-04-22T07:04:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.937+0000 7f41527fc640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f414006c8f0 0x7f414006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.937+0000 7f41527fc640 1 -- 172.21.15.44:0/2520623681 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f415808bd40 con 0x7f415c0caec0 2024-04-22T07:04:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.937+0000 7f41617de640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f414006c8f0 0x7f414006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:45.936 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.937+0000 7f41617de640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f414006c8f0 0x7f414006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f415c151300 tx=0x7f414c007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:45.939 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:45.941+0000 7f41527fc640 1 -- 172.21.15.44:0/2520623681 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4158059b10 con 0x7f415c0caec0 2024-04-22T07:04:46.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.166+0000 7f4164243640 1 -- 172.21.15.44:0/2520623681 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f415c0ca970 con 0x7f415c0caec0 2024-04-22T07:04:46.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.167+0000 7f41527fc640 1 -- 172.21.15.44:0/2520623681 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6908 (secure 0 0 0) 0x7f415805de10 con 0x7f415c0caec0 2024-04-22T07:04:46.165 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:46.165 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:04:44.948538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4324,"mds_1":4303,"mds_2":4300,"mds_3":4278,"mds_4":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"b","rank":3,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4324,"qdb_cluster":[4275,4278,4300,4303,4324]},"id":1}]} 2024-04-22T07:04:46.165 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-04-22T07:04:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 -- 172.21.15.44:0/2520623681 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f414006c8f0 msgr2=0x7f414006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f414006c8f0 0x7f414006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f415c151300 tx=0x7f414c007d50 comp rx=0 tx=0).stop 2024-04-22T07:04:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 -- 172.21.15.44:0/2520623681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f415c0caec0 msgr2=0x7f415c15a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f415c0caec0 0x7f415c15a520 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f415800ec10 tx=0x7f415800c520 comp rx=0 tx=0).stop 2024-04-22T07:04:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 -- 172.21.15.44:0/2520623681 shutdown_connections 2024-04-22T07:04:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f414006c8f0 0x7f414006edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f415c155120 0x7f415c14aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f415c15e8b0 0x7f415c151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.167 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 --2- 172.21.15.44:0/2520623681 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f415c0caec0 0x7f415c15a520 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 -- 172.21.15.44:0/2520623681 >> 172.21.15.44:0/2520623681 conn(0x7f415c0676b0 msgr2=0x7f415c0c5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 -- 172.21.15.44:0/2520623681 shutdown_connections 2024-04-22T07:04:46.168 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.169+0000 7f4164243640 1 -- 172.21.15.44:0/2520623681 wait complete. 2024-04-22T07:04:46.782 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-22T07:04:13.144503+0000', 'modified': '2024-04-22T07:04:44.948538+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4324, 'mds_1': 4303, 'mds_2': 4300, 'mds_3': 4278, 'mds_4': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'd', 'rank': 4, 'incarnation': 16, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.70:6834/1969018412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 1969018412}, {'type': 'v1', 'addr': '172.21.15.70:6834', 'nonce': 1969018412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4278': {'gid': 4278, 'name': 'b', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.70:6835/4058009549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6833', 'nonce': 4058009549}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 4058009549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4300': {'gid': 4300, 'name': 'c', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.44:6838/1520421043', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 1520421043}, {'type': 'v1', 'addr': '172.21.15.44:6838', 'nonce': 1520421043}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4303': {'gid': 4303, 'name': 'e', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.44:6839/1911557586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6837', 'nonce': 1911557586}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 1911557586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4324': {'gid': 4324, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.70:6837/2872131879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6836', 'nonce': 2872131879}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 2872131879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4324, 'qdb_cluster': [4275, 4278, 4300, 4303, 4324]} 2024-04-22T07:04:46.782 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T07:04:46.782 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-22T07:04:46.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.898+0000 7ff96b784640 1 Processor -- start 2024-04-22T07:04:46.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.899+0000 7ff96b784640 1 -- start start 2024-04-22T07:04:46.897 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.899+0000 7ff96b784640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff964154f50 0x7ff96415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.899+0000 7ff96b784640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff9640ccf30 0x7ff9640cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.899+0000 7ff96b784640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96415bc20 con 0x7ff9640cc880 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.899+0000 7ff96b784640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7ff96415f1b0 con 0x7ff9640ccf30 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.899+0000 7ff96b784640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7ff9640cb7c0 con 0x7ff964154f50 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.900+0000 7ff969d21640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff9640ccf30 0x7ff9640cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.900+0000 7ff969520640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff964154f50 0x7ff96415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.900+0000 7ff969d21640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff9640ccf30 0x7ff9640cd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46626/0 (socket says 172.21.15.44:46626) 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.900+0000 7ff969d21640 1 -- 172.21.15.44:0/99061862 learned_addr learned my addr 172.21.15.44:0/99061862 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.900+0000 7ff968d1f640 1 --1- 172.21.15.44:0/99061862 >> v1:172.21.15.70:6789/0 conn(0x7ff9640cc880 0x7ff9640c54c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:54848/0 (socket says 172.21.15.44:54848) 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.900+0000 7ff969520640 1 -- 172.21.15.44:0/99061862 >> v2:172.21.15.44:3300/0 conn(0x7ff9640ccf30 msgr2=0x7ff9640cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.900+0000 7ff969520640 1 --2- 172.21.15.44:0/99061862 >> v2:172.21.15.44:3300/0 conn(0x7ff9640ccf30 0x7ff9640cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.900+0000 7ff969520640 1 -- 172.21.15.44:0/99061862 >> v1:172.21.15.70:6789/0 conn(0x7ff9640cc880 legacy=0x7ff9640c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.900+0000 7ff969520640 1 -- 172.21.15.44:0/99061862 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96415e3f0 con 0x7ff964154f50 2024-04-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.900+0000 7ff95bfff640 1 -- 172.21.15.44:0/99061862 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767872516 0 0) 0x7ff954001110 con 0x7ff9640cc880 2024-04-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.901+0000 7ff969520640 1 --2- 172.21.15.44:0/99061862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff964154f50 0x7ff96415b6e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9640cb150 tx=0x7ff96002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5255808d0d80aacf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.901+0000 7ff95bfff640 1 -- 172.21.15.44:0/99061862 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960034070 con 0x7ff964154f50 2024-04-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.901+0000 7ff95bfff640 1 -- 172.21.15.44:0/99061862 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff960030610 con 0x7ff964154f50 2024-04-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.901+0000 7ff95bfff640 1 -- 172.21.15.44:0/99061862 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9600309e0 con 0x7ff964154f50 2024-04-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.901+0000 7ff96b784640 1 -- 172.21.15.44:0/99061862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff964154f50 msgr2=0x7ff96415b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.901+0000 7ff96b784640 1 --2- 172.21.15.44:0/99061862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff964154f50 0x7ff96415b6e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9640cb150 tx=0x7ff96002b960 comp rx=0 tx=0).stop 2024-04-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.901+0000 7ff96b784640 1 -- 172.21.15.44:0/99061862 shutdown_connections 2024-04-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.901+0000 7ff96b784640 1 --2- 172.21.15.44:0/99061862 >> v2:172.21.15.44:3300/0 conn(0x7ff9640ccf30 0x7ff9640cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.901+0000 7ff96b784640 1 --2- 172.21.15.44:0/99061862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff964154f50 0x7ff96415b6e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.901+0000 7ff96b784640 1 -- 172.21.15.44:0/99061862 >> 172.21.15.44:0/99061862 conn(0x7ff964067d40 msgr2=0x7ff96406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.902+0000 7ff96b784640 1 -- 172.21.15.44:0/99061862 shutdown_connections 2024-04-22T07:04:46.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.902+0000 7ff96b784640 1 -- 172.21.15.44:0/99061862 wait complete. 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.903+0000 7ff96b784640 1 Processor -- start 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.903+0000 7ff96b784640 1 -- start start 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.903+0000 7ff96b784640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9640cc880 0x7ff9641581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.903+0000 7ff96b784640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9640ccf30 0x7ff9641f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff96b784640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff964154f50 0x7ff9641f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff96b784640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff96415ec90 con 0x7ff9640ccf30 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff96b784640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff96415e9b0 con 0x7ff964154f50 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff96b784640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff964155330 con 0x7ff9640cc880 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff968d1f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9640ccf30 0x7ff9641f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff968d1f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9640ccf30 0x7ff9641f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46632/0 (socket says 172.21.15.44:46632) 2024-04-22T07:04:46.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff968d1f640 1 -- 172.21.15.44:0/3728684524 learned_addr learned my addr 172.21.15.44:0/3728684524 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff969520640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9640cc880 0x7ff9641581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff969d21640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff964154f50 0x7ff9641f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff968d1f640 1 -- 172.21.15.44:0/3728684524 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9640cc880 msgr2=0x7ff9641581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff968d1f640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9640cc880 0x7ff9641581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff968d1f640 1 -- 172.21.15.44:0/3728684524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff964154f50 msgr2=0x7ff9641f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff968d1f640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff964154f50 0x7ff9641f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff968d1f640 1 -- 172.21.15.44:0/3728684524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff964157c40 con 0x7ff9640ccf30 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff969520640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9640cc880 0x7ff9641581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff968d1f640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9640ccf30 0x7ff9641f4390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c7b0 tx=0x7ff95400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff959ffb640 1 -- 172.21.15.44:0/3728684524 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954018070 con 0x7ff9640ccf30 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff959ffb640 1 -- 172.21.15.44:0/3728684524 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9540072d0 con 0x7ff9640ccf30 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff96b784640 1 -- 172.21.15.44:0/3728684524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff964157ee0 con 0x7ff9640ccf30 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.904+0000 7ff959ffb640 1 -- 172.21.15.44:0/3728684524 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9540047e0 con 0x7ff9640ccf30 2024-04-22T07:04:46.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.905+0000 7ff96b784640 1 -- 172.21.15.44:0/3728684524 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff964155a50 con 0x7ff9640ccf30 2024-04-22T07:04:46.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.906+0000 7ff959ffb640 1 -- 172.21.15.44:0/3728684524 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff95401b070 con 0x7ff9640ccf30 2024-04-22T07:04:46.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.907+0000 7ff959ffb640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.907+0000 7ff959ffb640 1 -- 172.21.15.44:0/3728684524 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff95408af20 con 0x7ff9640ccf30 2024-04-22T07:04:46.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.907+0000 7ff969520640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.907+0000 7ff969520640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff94806c8f0 0x7ff94806edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff960036000 tx=0x7ff96003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff96e379740 1 Processor -- start 2024-04-22T07:04:46.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff96e379740 1 -- start start 2024-04-22T07:04:46.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff96e379740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d4ae40 0x5615e0d53f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff96e379740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 0x5615e0d544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff96e379740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d07ca0 0x5615e0d2fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff96e379740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615e0c6eb10 con 0x5615e0d07ca0 2024-04-22T07:04:46.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff96e379740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5615e0c0c240 con 0x5615e0d4ae40 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff96e379740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5615e0c6aad0 con 0x5615e0d0bec0 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff969d21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d07ca0 0x5615e0d2fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff968d1f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 0x5615e0d544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff969520640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d4ae40 0x5615e0d53f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff969d21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d07ca0 0x5615e0d2fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46642/0 (socket says 172.21.15.44:46642) 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.908+0000 7ff969d21640 1 -- 172.21.15.44:0/401329811 learned_addr learned my addr 172.21.15.44:0/401329811 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff968d1f640 1 -- 172.21.15.44:0/401329811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d4ae40 msgr2=0x5615e0d53f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff968d1f640 1 --2- 172.21.15.44:0/401329811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d4ae40 0x5615e0d53f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff968d1f640 1 -- 172.21.15.44:0/401329811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d07ca0 msgr2=0x5615e0d2fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff968d1f640 1 --2- 172.21.15.44:0/401329811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d07ca0 0x5615e0d2fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff968d1f640 1 -- 172.21.15.44:0/401329811 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e09812d0 con 0x5615e0d0bec0 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff968d1f640 1 --2- 172.21.15.44:0/401329811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 0x5615e0d544a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff954089380 tx=0x7ff954001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d949e65ba04119f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff94effd640 1 -- 172.21.15.44:0/401329811 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954061790 con 0x5615e0d0bec0 2024-04-22T07:04:46.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff94effd640 1 -- 172.21.15.44:0/401329811 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff954059e50 con 0x5615e0d0bec0 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff94effd640 1 -- 172.21.15.44:0/401329811 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954055240 con 0x5615e0d0bec0 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff96e379740 1 -- 172.21.15.44:0/401329811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 msgr2=0x5615e0d544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.909+0000 7ff96e379740 1 --2- 172.21.15.44:0/401329811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 0x5615e0d544a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff954089380 tx=0x7ff954001e70 comp rx=0 tx=0).stop 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.910+0000 7ff96e379740 1 -- 172.21.15.44:0/401329811 shutdown_connections 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.910+0000 7ff96e379740 1 --2- 172.21.15.44:0/401329811 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 0x5615e0d544a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.910+0000 7ff96e379740 1 --2- 172.21.15.44:0/401329811 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d07ca0 0x5615e0d2fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.910+0000 7ff96e379740 1 --2- 172.21.15.44:0/401329811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d4ae40 0x5615e0d53f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.910+0000 7ff96e379740 1 -- 172.21.15.44:0/401329811 >> 172.21.15.44:0/401329811 conn(0x5615e0d06990 msgr2=0x5615e0d09290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.910+0000 7ff96e379740 1 -- 172.21.15.44:0/401329811 shutdown_connections 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.910+0000 7ff96e379740 1 -- 172.21.15.44:0/401329811 wait complete. 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.910+0000 7ff96e379740 1 -- start start 2024-04-22T07:04:46.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.910+0000 7ff96e379740 1 Processor -- start 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff96e379740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 0x5615e0d530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff96e379740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d5f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff96e379740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d5f8d0 0x5615e0d44650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff96e379740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615e0d0a250 con 0x5615e0d4ae40 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff96e379740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5615e0bf7190 con 0x5615e0d5f8d0 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff96e379740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5615e0d0a530 con 0x5615e0d0bec0 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff96e379740 10 client.0 _ll_register_callbacks cb 0x5615e0bf28b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff968d1f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d5f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff968d1f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d5f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46656/0 (socket says 172.21.15.44:46656) 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff968d1f640 1 -- 172.21.15.44:0/4037936748 learned_addr learned my addr 172.21.15.44:0/4037936748 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff969520640 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 0x5615e0d530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.911+0000 7ff969d21640 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d5f8d0 0x5615e0d44650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff968d1f640 1 -- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 msgr2=0x5615e0d530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff968d1f640 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 0x5615e0d530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff968d1f640 1 -- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d5f8d0 msgr2=0x5615e0d44650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff968d1f640 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d5f8d0 0x5615e0d44650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff968d1f640 1 -- 172.21.15.44:0/4037936748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e0a4b6b0 con 0x5615e0d4ae40 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff968d1f640 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d5f390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff954000f60 tx=0x7ff954000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb3732f53d35a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff943fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff943fff640 1 -- 172.21.15.44:0/4037936748 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954077060 con 0x5615e0d4ae40 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff943fff640 1 -- 172.21.15.44:0/4037936748 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff954058020 con 0x5615e0d4ae40 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff96e379740 1 -- 172.21.15.44:0/4037936748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff954059e50 con 0x5615e0d4ae40 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff943fff640 1 -- 172.21.15.44:0/4037936748 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff954061790 con 0x5615e0d4ae40 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff96e379740 10 client.4653 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff96e379740 1 -- 172.21.15.44:0/4037936748 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5615e0d5fd50 con 0x5615e0d4ae40 2024-04-22T07:04:46.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff943fff640 1 -- 172.21.15.44:0/4037936748 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff954059e50 con 0x5615e0d4ae40 2024-04-22T07:04:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff943fff640 1 -- 172.21.15.44:0/4037936748 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7ff954054920 con 0x5615e0d4ae40 2024-04-22T07:04:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff96e379740 10 client.4653 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff96e379740 10 client.4653 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-22T07:04:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff96e379740 20 client.4653 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff96e379740 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x5615e0d685c0 0x5615e0d6a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff96e379740 4 client.4653 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.912+0000 7ff96e379740 1 -- 172.21.15.44:0/4037936748 --> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615e0c7c530 con 0x5615e0d685c0 2024-04-22T07:04:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.913+0000 7ff969520640 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x5615e0d685c0 0x5615e0d6a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.913+0000 7ff969520640 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x5615e0d685c0 0x5615e0d6a9b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bab81aa8aedefc92 server_cookie=e0097092975939ca in_seq=0 out_seq=0 2024-04-22T07:04:46.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:46.913+0000 7ff943fff640 10 client.4653 ms_handle_connect on v2:172.21.15.70:6832/1969018412 2024-04-22T07:04:49.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff943fff640 1 -- 172.21.15.44:0/4037936748 <== mds.4 v2:172.21.15.70:6832/1969018412 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615e0c7c530 con 0x5615e0d685c0 2024-04-22T07:04:49.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff943fff640 10 client.4653 handle_command_reply: tid=0 2024-04-22T07:04:49.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 client.4653 shutdown 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 -- 172.21.15.44:0/4037936748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 msgr2=0x5615e0d5f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d5f390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff954000f60 tx=0x7ff954000ed0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 -- 172.21.15.44:0/4037936748 shutdown_connections 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x5615e0d685c0 0x5615e0d6a9b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d5f8d0 0x5615e0d44650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff943fff640 0 client.4653 ms_handle_reset on v2:172.21.15.70:6832/1969018412 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d5f390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 --2- 172.21.15.44:0/4037936748 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d0bec0 0x5615e0d530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 -- 172.21.15.44:0/4037936748 >> 172.21.15.44:0/4037936748 conn(0x5615e0d07ca0 msgr2=0x5615e0d0aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 -- 172.21.15.44:0/4037936748 shutdown_connections 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 1 -- 172.21.15.44:0/4037936748 wait complete. 2024-04-22T07:04:49.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.951+0000 7ff96e379740 20 client.4653 trim_cache size 0 max 16384 2024-04-22T07:04:49.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.972+0000 7ff96e379740 1 Processor -- start 2024-04-22T07:04:49.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.972+0000 7ff96e379740 1 -- start start 2024-04-22T07:04:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.972+0000 7ff96e379740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d07ca0 0x5615e0d2be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.972+0000 7ff96e379740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d0bec0 0x5615e0d47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.972+0000 7ff96e379740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d4ae40 0x5615e0d47980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.972+0000 7ff96e379740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff964156810 con 0x5615e0d0bec0 2024-04-22T07:04:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.972+0000 7ff96e379740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5615e0bf7190 con 0x5615e0d07ca0 2024-04-22T07:04:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.972+0000 7ff96e379740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5615e0d0a250 con 0x5615e0d4ae40 2024-04-22T07:04:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.972+0000 7ff968d1f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d0bec0 0x5615e0d47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:49.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff968d1f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d0bec0 0x5615e0d47440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46666/0 (socket says 172.21.15.44:46666) 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff968d1f640 1 -- 172.21.15.44:0/3175563123 learned_addr learned my addr 172.21.15.44:0/3175563123 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff969d21640 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d4ae40 0x5615e0d47980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff969520640 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d07ca0 0x5615e0d2be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff968d1f640 1 -- 172.21.15.44:0/3175563123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d4ae40 msgr2=0x5615e0d47980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff968d1f640 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d4ae40 0x5615e0d47980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff968d1f640 1 -- 172.21.15.44:0/3175563123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d07ca0 msgr2=0x5615e0d2be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff968d1f640 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d07ca0 0x5615e0d2be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff968d1f640 1 -- 172.21.15.44:0/3175563123 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e0d051c0 con 0x5615e0d0bec0 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff969520640 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d07ca0 0x5615e0d2be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff969d21640 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d4ae40 0x5615e0d47980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff968d1f640 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d0bec0 0x5615e0d47440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff954055030 tx=0x7ff954084070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c3c8a83a4a0870 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff94e7fc640 1 -- 172.21.15.44:0/3175563123 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9540589a0 con 0x5615e0d0bec0 2024-04-22T07:04:49.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff94e7fc640 1 -- 172.21.15.44:0/3175563123 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff954059e50 con 0x5615e0d0bec0 2024-04-22T07:04:49.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.973+0000 7ff94e7fc640 1 -- 172.21.15.44:0/3175563123 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954058240 con 0x5615e0d0bec0 2024-04-22T07:04:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.974+0000 7ff96e379740 1 -- 172.21.15.44:0/3175563123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d0bec0 msgr2=0x5615e0d47440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.974+0000 7ff96e379740 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d0bec0 0x5615e0d47440 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff954055030 tx=0x7ff954084070 comp rx=0 tx=0).stop 2024-04-22T07:04:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.974+0000 7ff96e379740 1 -- 172.21.15.44:0/3175563123 shutdown_connections 2024-04-22T07:04:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.974+0000 7ff96e379740 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d4ae40 0x5615e0d47980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.974+0000 7ff96e379740 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d0bec0 0x5615e0d47440 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.974+0000 7ff96e379740 1 --2- 172.21.15.44:0/3175563123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d07ca0 0x5615e0d2be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.974+0000 7ff96e379740 1 -- 172.21.15.44:0/3175563123 >> 172.21.15.44:0/3175563123 conn(0x5615e0d4a170 msgr2=0x5615e0d441f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.974+0000 7ff96e379740 1 -- 172.21.15.44:0/3175563123 shutdown_connections 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.974+0000 7ff96e379740 1 -- 172.21.15.44:0/3175563123 wait complete. 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.975+0000 7ff96e379740 1 -- start start 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.975+0000 7ff96e379740 1 Processor -- start 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff96e379740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d07ca0 0x5615e0dc69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff96e379740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d0bec0 0x5615e0d0a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff96e379740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d4bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff96e379740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5615e0d53900 con 0x5615e0d4ae40 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff96e379740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5615e0d4efb0 con 0x5615e0d0bec0 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff96e379740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5615e0d4d600 con 0x5615e0d07ca0 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff96e379740 10 client.0 _ll_register_callbacks cb 0x7ff95c009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff969d21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d4bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:49.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff969d21640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d4bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46674/0 (socket says 172.21.15.44:46674) 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff969d21640 1 -- 172.21.15.44:0/271885224 learned_addr learned my addr 172.21.15.44:0/271885224 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff969520640 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d07ca0 0x5615e0dc69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.976+0000 7ff968d1f640 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d0bec0 0x5615e0d0a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff969d21640 1 -- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d07ca0 msgr2=0x5615e0dc69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff969d21640 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d07ca0 0x5615e0dc69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff969d21640 1 -- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d0bec0 msgr2=0x5615e0d0a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff969d21640 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d0bec0 0x5615e0d0a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff969d21640 1 -- 172.21.15.44:0/271885224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e0d0b180 con 0x5615e0d4ae40 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff968d1f640 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d0bec0 0x5615e0d0a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff969d21640 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d4bb10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5615e0d04f30 tx=0x7ff95c017f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5411f2317c20b3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff943fff640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff943fff640 1 -- 172.21.15.44:0/271885224 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c007d50 con 0x5615e0d4ae40 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff943fff640 1 -- 172.21.15.44:0/271885224 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff95c022ad0 con 0x5615e0d4ae40 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff96e379740 1 -- 172.21.15.44:0/271885224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff954059e50 con 0x5615e0d4ae40 2024-04-22T07:04:49.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff943fff640 1 -- 172.21.15.44:0/271885224 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff95c022cd0 con 0x5615e0d4ae40 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff96e379740 10 client.4659 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.977+0000 7ff96e379740 1 -- 172.21.15.44:0/271885224 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5615e0d4ebf0 con 0x5615e0d4ae40 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff943fff640 1 -- 172.21.15.44:0/271885224 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff954059e50 con 0x5615e0d4ae40 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff943fff640 1 -- 172.21.15.44:0/271885224 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7ff95c029050 con 0x5615e0d4ae40 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff96e379740 10 client.4659 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff96e379740 10 client.4659 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff96e379740 20 client.4659 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff96e379740 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x5615e0d5f8d0 0x5615e0d659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff96e379740 4 client.4659 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff96e379740 1 -- 172.21.15.44:0/271885224 --> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] -- command(tid 0: {"prefix": "status"}) -- 0x5615e0c763a0 con 0x5615e0d5f8d0 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff969520640 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x5615e0d5f8d0 0x5615e0d659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff943fff640 10 client.4659 ms_handle_connect on v2:172.21.15.70:6832/1969018412 2024-04-22T07:04:49.980 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.978+0000 7ff969520640 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x5615e0d5f8d0 0x5615e0d659c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c8ffb842f544339 server_cookie=25972151b3bcffea in_seq=0 out_seq=0 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff943fff640 1 -- 172.21.15.44:0/271885224 <== mds.4 v2:172.21.15.70:6832/1969018412 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5615e0c763a0 con 0x5615e0d5f8d0 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff943fff640 10 client.4659 handle_command_reply: tid=0 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 client.4659 shutdown 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 -- 172.21.15.44:0/271885224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 msgr2=0x5615e0d4bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d4bb10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5615e0d04f30 tx=0x7ff95c017f60 comp rx=0 tx=0).stop 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 -- 172.21.15.44:0/271885224 shutdown_connections 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x5615e0d5f8d0 0x5615e0d659c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5615e0d4ae40 0x5615e0d4bb10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5615e0d0bec0 0x5615e0d0a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff943fff640 0 client.4659 ms_handle_reset on v2:172.21.15.70:6832/1969018412 2024-04-22T07:04:49.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 --2- 172.21.15.44:0/271885224 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5615e0d07ca0 0x5615e0dc69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 -- 172.21.15.44:0/271885224 >> 172.21.15.44:0/271885224 conn(0x5615e0d4a170 msgr2=0x5615e0d441f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:49.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 -- 172.21.15.44:0/271885224 shutdown_connections 2024-04-22T07:04:49.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 1 -- 172.21.15.44:0/271885224 wait complete. 2024-04-22T07:04:49.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.979+0000 7ff96e379740 20 client.4659 trim_cache size 0 max 16384 2024-04-22T07:04:49.982 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:04:49.982 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "dc21e26c-6c65-477f-a367-ffb001b78b47", 2024-04-22T07:04:49.982 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 4, 2024-04-22T07:04:49.982 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4275, 2024-04-22T07:04:49.982 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T07:04:49.982 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T07:04:49.983 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T07:04:49.983 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 10.042369115, 2024-04-22T07:04:49.983 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:49.983 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 23, 2024-04-22T07:04:49.983 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 23, 2024-04-22T07:04:49.983 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 42.837472599000002 2024-04-22T07:04:49.983 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 -- 172.21.15.44:0/3728684524 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff94806c8f0 msgr2=0x7ff94806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff94806c8f0 0x7ff94806edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff960036000 tx=0x7ff96003b040 comp rx=0 tx=0).stop 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 -- 172.21.15.44:0/3728684524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9640ccf30 msgr2=0x7ff9641f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9640ccf30 0x7ff9641f4390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c7b0 tx=0x7ff95400cc70 comp rx=0 tx=0).stop 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 -- 172.21.15.44:0/3728684524 shutdown_connections 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff94806c8f0 0x7ff94806edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff964154f50 0x7ff9641f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff9640ccf30 0x7ff9641f4390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 --2- 172.21.15.44:0/3728684524 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff9640cc880 0x7ff9641581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 -- 172.21.15.44:0/3728684524 >> 172.21.15.44:0/3728684524 conn(0x7ff964067d40 msgr2=0x7ff964071f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 -- 172.21.15.44:0/3728684524 shutdown_connections 2024-04-22T07:04:49.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:49.980+0000 7ff96b784640 1 -- 172.21.15.44:0/3728684524 wait complete. 2024-04-22T07:04:49.991 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T07:04:50.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.102+0000 7f1f94e11640 1 Processor -- start 2024-04-22T07:04:50.100 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.102+0000 7f1f94e11640 1 -- start start 2024-04-22T07:04:50.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f94e11640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f9015add0 0x7f1f900cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f94e11640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9015b280 con 0x7f1f90071950 2024-04-22T07:04:50.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f94e11640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9015b560 con 0x7f1f90071d30 2024-04-22T07:04:50.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f94e11640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f1f900cbf20 con 0x7f1f9015add0 2024-04-22T07:04:50.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f8e59c640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f1f90071950 0x7f1f900728f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:32970/0 (socket says 172.21.15.44:32970) 2024-04-22T07:04:50.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f8e59c640 1 -- 172.21.15.44:0/1804307218 learned_addr learned my addr 172.21.15.44:0/1804307218 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:50.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f87fff640 1 --2- 172.21.15.44:0/1804307218 >> v2:172.21.15.70:3300/0 conn(0x7f1f9015add0 0x7f1f900cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f8dd9b640 1 -- 172.21.15.44:0/1804307218 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2344961682 0 0) 0x7f1f9015b280 con 0x7f1f90071950 2024-04-22T07:04:50.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f8dd9b640 1 -- 172.21.15.44:0/1804307218 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f7c003520 con 0x7f1f90071950 2024-04-22T07:04:50.102 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f8dd9b640 1 -- 172.21.15.44:0/1804307218 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1507005611 0 0) 0x7f1f9015b560 con 0x7f1f90071d30 2024-04-22T07:04:50.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.103+0000 7f1f8dd9b640 1 -- 172.21.15.44:0/1804307218 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f9015b280 con 0x7f1f90071d30 2024-04-22T07:04:50.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.104+0000 7f1f87fff640 1 -- 172.21.15.44:0/1804307218 >> v1:172.21.15.70:6789/0 conn(0x7f1f90071d30 legacy=0x7f1f90154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.104+0000 7f1f8dd9b640 1 -- 172.21.15.44:0/1804307218 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3361931197 0 0) 0x7f1f7c003520 con 0x7f1f90071950 2024-04-22T07:04:50.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.104+0000 7f1f87fff640 1 -- 172.21.15.44:0/1804307218 >> v1:172.21.15.44:6789/0 conn(0x7f1f90071950 legacy=0x7f1f900728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.104+0000 7f1f87fff640 1 -- 172.21.15.44:0/1804307218 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9015b840 con 0x7f1f9015add0 2024-04-22T07:04:50.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.104+0000 7f1f87fff640 1 --2- 172.21.15.44:0/1804307218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f9015add0 0x7f1f900cc680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1f780097a0 tx=0x7f1f7802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51f0bac8df2c33c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.105+0000 7f1f8dd9b640 1 -- 172.21.15.44:0/1804307218 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78037070 con 0x7f1f9015add0 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.105+0000 7f1f8dd9b640 1 -- 172.21.15.44:0/1804307218 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f78030510 con 0x7f1f9015add0 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.105+0000 7f1f8dd9b640 1 -- 172.21.15.44:0/1804307218 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80001690 con 0x7f1f9015add0 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.105+0000 7f1f94e11640 1 -- 172.21.15.44:0/1804307218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f9015add0 msgr2=0x7f1f900cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.105+0000 7f1f94e11640 1 --2- 172.21.15.44:0/1804307218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f9015add0 0x7f1f900cc680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1f780097a0 tx=0x7f1f7802b790 comp rx=0 tx=0).stop 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.105+0000 7f1f94e11640 1 -- 172.21.15.44:0/1804307218 shutdown_connections 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.105+0000 7f1f94e11640 1 --2- 172.21.15.44:0/1804307218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f9015add0 0x7f1f900cc680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.105+0000 7f1f94e11640 1 -- 172.21.15.44:0/1804307218 >> 172.21.15.44:0/1804307218 conn(0x7f1f9006b970 msgr2=0x7f1f9006bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.105+0000 7f1f94e11640 1 -- 172.21.15.44:0/1804307218 shutdown_connections 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.105+0000 7f1f94e11640 1 -- 172.21.15.44:0/1804307218 wait complete. 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.107+0000 7f1f94e11640 1 Processor -- start 2024-04-22T07:04:50.105 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.107+0000 7f1f94e11640 1 -- start start 2024-04-22T07:04:50.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.107+0000 7f1f94e11640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f90071950 0x7f1f90145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.107+0000 7f1f94e11640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f90071d30 0x7f1f900c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.107+0000 7f1f94e11640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f9015add0 0x7f1f900c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.107+0000 7f1f94e11640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f900721e0 con 0x7f1f90071d30 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.107+0000 7f1f94e11640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1f900cc1f0 con 0x7f1f9015add0 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.107+0000 7f1f94e11640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f1f90072360 con 0x7f1f90071950 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f90071d30 0x7f1f900c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f90071d30 0x7f1f900c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46676/0 (socket says 172.21.15.44:46676) 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f87fff640 1 -- 172.21.15.44:0/2137317796 learned_addr learned my addr 172.21.15.44:0/2137317796 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f90071950 0x7f1f90145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f8ed9d640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f9015add0 0x7f1f900c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f87fff640 1 -- 172.21.15.44:0/2137317796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f90071950 msgr2=0x7f1f90145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f87fff640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f90071950 0x7f1f90145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f87fff640 1 -- 172.21.15.44:0/2137317796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f9015add0 msgr2=0x7f1f900c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f87fff640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f9015add0 0x7f1f900c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f87fff640 1 -- 172.21.15.44:0/2137317796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f901f0a10 con 0x7f1f90071d30 2024-04-22T07:04:50.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f87fff640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f90071d30 0x7f1f900c3f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1f78035040 tx=0x7f1f780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f86ffd640 1 -- 172.21.15.44:0/2137317796 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78042020 con 0x7f1f90071d30 2024-04-22T07:04:50.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f86ffd640 1 -- 172.21.15.44:0/2137317796 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f780044c0 con 0x7f1f90071d30 2024-04-22T07:04:50.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f94e11640 1 -- 172.21.15.44:0/2137317796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f901f0cd0 con 0x7f1f90071d30 2024-04-22T07:04:50.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f86ffd640 1 -- 172.21.15.44:0/2137317796 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78037040 con 0x7f1f90071d30 2024-04-22T07:04:50.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.108+0000 7f1f94e11640 1 -- 172.21.15.44:0/2137317796 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f901f1290 con 0x7f1f90071d30 2024-04-22T07:04:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.110+0000 7f1f86ffd640 1 -- 172.21.15.44:0/2137317796 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f78045020 con 0x7f1f90071d30 2024-04-22T07:04:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.111+0000 7f1f86ffd640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1f7006c920 0x7f1f7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.111+0000 7f1f86ffd640 1 -- 172.21.15.44:0/2137317796 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1f780ad7a0 con 0x7f1f90071d30 2024-04-22T07:04:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.111+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1f7006c920 0x7f1f7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.111+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1f7006c920 0x7f1f7006ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000c00 tx=0x7f1f80001550 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.111+0000 7f1f97a00740 1 Processor -- start 2024-04-22T07:04:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f97a00740 1 -- start start 2024-04-22T07:04:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbae40 0x555ecedc3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555eced77ca0 0x555eced9fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f97a00740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555ececdeb10 con 0x555eced77ca0 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f97a00740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555ecec7c240 con 0x555ecedbae40 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f97a00740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555ececdaad0 con 0x555eced7bec0 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f8ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555eced77ca0 0x555eced9fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f8ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555eced77ca0 0x555eced9fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46690/0 (socket says 172.21.15.44:46690) 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f8ed9d640 1 -- 172.21.15.44:0/775603371 learned_addr learned my addr 172.21.15.44:0/775603371 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/775603371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbae40 0x555ecedc3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.112+0000 7f1f87fff640 1 --2- 172.21.15.44:0/775603371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.113+0000 7f1f8ed9d640 1 -- 172.21.15.44:0/775603371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 msgr2=0x555ecedc44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.113+0000 7f1f8ed9d640 1 --2- 172.21.15.44:0/775603371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.113+0000 7f1f8ed9d640 1 -- 172.21.15.44:0/775603371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbae40 msgr2=0x555ecedc3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.113+0000 7f1f8ed9d640 1 --2- 172.21.15.44:0/775603371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbae40 0x555ecedc3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.113+0000 7f1f8ed9d640 1 -- 172.21.15.44:0/775603371 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ece9f12d0 con 0x555eced77ca0 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.113+0000 7f1f8ed9d640 1 --2- 172.21.15.44:0/775603371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555eced77ca0 0x555eced9fee0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1f88001c80 tx=0x7f1f88002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7efc0783acfd16 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.113+0000 7f1f6ffff640 1 -- 172.21.15.44:0/775603371 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88019070 con 0x555eced77ca0 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.113+0000 7f1f6ffff640 1 -- 172.21.15.44:0/775603371 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f88003b80 con 0x555eced77ca0 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.113+0000 7f1f6ffff640 1 -- 172.21.15.44:0/775603371 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800e8c0 con 0x555eced77ca0 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 -- 172.21.15.44:0/775603371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555eced77ca0 msgr2=0x555eced9fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 --2- 172.21.15.44:0/775603371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555eced77ca0 0x555eced9fee0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1f88001c80 tx=0x7f1f88002140 comp rx=0 tx=0).stop 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 -- 172.21.15.44:0/775603371 shutdown_connections 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 --2- 172.21.15.44:0/775603371 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 --2- 172.21.15.44:0/775603371 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555eced77ca0 0x555eced9fee0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 --2- 172.21.15.44:0/775603371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbae40 0x555ecedc3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 -- 172.21.15.44:0/775603371 >> 172.21.15.44:0/775603371 conn(0x555eced76990 msgr2=0x555eced79290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 -- 172.21.15.44:0/775603371 shutdown_connections 2024-04-22T07:04:50.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 -- 172.21.15.44:0/775603371 wait complete. 2024-04-22T07:04:50.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 -- start start 2024-04-22T07:04:50.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.114+0000 7f1f97a00740 1 Processor -- start 2024-04-22T07:04:50.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555eced74460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbff10 0x555ecedb4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f97a00740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555eced7a250 con 0x555ecedbae40 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f97a00740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555ecec67190 con 0x555ecedbff10 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f97a00740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555eced7a530 con 0x555eced7bec0 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f97a00740 10 client.0 _ll_register_callbacks cb 0x555ecec628b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555eced74460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555eced74460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46694/0 (socket says 172.21.15.44:46694) 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f87fff640 1 -- 172.21.15.44:0/991137544 learned_addr learned my addr 172.21.15.44:0/991137544 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.115+0000 7f1f8ed9d640 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbff10 0x555ecedb4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f87fff640 1 -- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 msgr2=0x555ecedc1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f87fff640 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f87fff640 1 -- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbff10 msgr2=0x555ecedb4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f87fff640 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbff10 0x555ecedb4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f87fff640 1 -- 172.21.15.44:0/991137544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eceabb6b0 con 0x555ecedbae40 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f87fff640 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555eced74460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f7807b1d0 tx=0x7f1f7807b090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b99418766d7b2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/991137544 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78083e20 con 0x555ecedbae40 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/991137544 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1f780a6ce0 con 0x555ecedbae40 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f97a00740 1 -- 172.21.15.44:0/991137544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f88003b80 con 0x555ecedbae40 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/991137544 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f1f7807f880 con 0x555ecedbae40 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f97a00740 10 client.4671 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f97a00740 1 -- 172.21.15.44:0/991137544 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x555eced749a0 con 0x555ecedbae40 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/991137544 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f88003b80 con 0x555ecedbae40 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/991137544 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f1f7807a920 con 0x555ecedbae40 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f97a00740 10 client.4671 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:50.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.116+0000 7f1f97a00740 10 client.4671 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.117+0000 7f1f97a00740 20 client.4671 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.117+0000 7f1f97a00740 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x555ecedcf1b0 0x555eceddaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.117+0000 7f1f97a00740 4 client.4671 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.117+0000 7f1f97a00740 1 -- 172.21.15.44:0/991137544 --> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ececec530 con 0x555ecedcf1b0 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.117+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x555ecedcf1b0 0x555eceddaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.117+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x555ecedcf1b0 0x555eceddaa60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4aae677bd55fb10c server_cookie=b69a6e9e4f41ddd0 in_seq=0 out_seq=0 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.117+0000 7f1f6d7fa640 10 client.4671 ms_handle_connect on v2:172.21.15.70:6833/4058009549 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/991137544 <== mds.3 v2:172.21.15.70:6833/4058009549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ececec530 con 0x555ecedcf1b0 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f6d7fa640 10 client.4671 handle_command_reply: tid=0 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 client.4671 shutdown 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 -- 172.21.15.44:0/991137544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 msgr2=0x555eced74460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555eced74460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f7807b1d0 tx=0x7f1f7807b090 comp rx=0 tx=0).stop 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 -- 172.21.15.44:0/991137544 shutdown_connections 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x555ecedcf1b0 0x555eceddaa60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.121 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbff10 0x555ecedb4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555eced74460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f6d7fa640 0 client.4671 ms_handle_reset on v2:172.21.15.70:6833/4058009549 2024-04-22T07:04:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 --2- 172.21.15.44:0/991137544 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 -- 172.21.15.44:0/991137544 >> 172.21.15.44:0/991137544 conn(0x555eced77ca0 msgr2=0x555ecedc3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 -- 172.21.15.44:0/991137544 shutdown_connections 2024-04-22T07:04:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 1 -- 172.21.15.44:0/991137544 wait complete. 2024-04-22T07:04:50.122 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.118+0000 7f1f97a00740 20 client.4671 trim_cache size 0 max 16384 2024-04-22T07:04:50.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.137+0000 7f1f97a00740 1 Processor -- start 2024-04-22T07:04:50.134 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.137+0000 7f1f97a00740 1 -- start start 2024-04-22T07:04:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.137+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eced7bec0 0x555eced7aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.137+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ecedbae40 0x555ecedc3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.137+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbff10 0x555ecedc4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.137+0000 7f1f97a00740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f1f901fd010 con 0x555ecedbff10 2024-04-22T07:04:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.137+0000 7f1f97a00740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9014bf00 con 0x555eced7bec0 2024-04-22T07:04:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.137+0000 7f1f97a00740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555eced9af40 con 0x555ecedbae40 2024-04-22T07:04:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f8ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbff10 0x555ecedc4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f8ed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbff10 0x555ecedc4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46710/0 (socket says 172.21.15.44:46710) 2024-04-22T07:04:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f8ed9d640 1 -- 172.21.15.44:0/4213481850 learned_addr learned my addr 172.21.15.44:0/4213481850 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.137 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/4213481850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eced7bec0 0x555eced7aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f87fff640 1 --2- 172.21.15.44:0/4213481850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ecedbae40 0x555ecedc3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f8ed9d640 1 -- 172.21.15.44:0/4213481850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ecedbae40 msgr2=0x555ecedc3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f8ed9d640 1 --2- 172.21.15.44:0/4213481850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ecedbae40 0x555ecedc3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f8ed9d640 1 -- 172.21.15.44:0/4213481850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eced7bec0 msgr2=0x555eced7aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f8ed9d640 1 --2- 172.21.15.44:0/4213481850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eced7bec0 0x555eced7aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f8ed9d640 1 -- 172.21.15.44:0/4213481850 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecedc1380 con 0x555ecedbff10 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f8ed9d640 1 --2- 172.21.15.44:0/4213481850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbff10 0x555ecedc4260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1f88007fd0 tx=0x7f1f8800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c0db8cfa1b02e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f6f7fe640 1 -- 172.21.15.44:0/4213481850 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88019040 con 0x555ecedbff10 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f6f7fe640 1 -- 172.21.15.44:0/4213481850 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f88024070 con 0x555ecedbff10 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.138+0000 7f1f6f7fe640 1 -- 172.21.15.44:0/4213481850 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800ed00 con 0x555ecedbff10 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 -- 172.21.15.44:0/4213481850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbff10 msgr2=0x555ecedc4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 --2- 172.21.15.44:0/4213481850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbff10 0x555ecedc4260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1f88007fd0 tx=0x7f1f8800a000 comp rx=0 tx=0).stop 2024-04-22T07:04:50.138 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 -- 172.21.15.44:0/4213481850 shutdown_connections 2024-04-22T07:04:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 --2- 172.21.15.44:0/4213481850 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555ecedbae40 0x555ecedc3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 --2- 172.21.15.44:0/4213481850 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbff10 0x555ecedc4260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 --2- 172.21.15.44:0/4213481850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555eced7bec0 0x555eced7aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 -- 172.21.15.44:0/4213481850 >> 172.21.15.44:0/4213481850 conn(0x555eced77ca0 msgr2=0x555eced79290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 -- 172.21.15.44:0/4213481850 shutdown_connections 2024-04-22T07:04:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 -- 172.21.15.44:0/4213481850 wait complete. 2024-04-22T07:04:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 -- start start 2024-04-22T07:04:50.139 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.139+0000 7f1f97a00740 1 Processor -- start 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555ecee32cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f97a00740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbff10 0x555ecee33200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f97a00740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555ecedc39a0 con 0x555ecedbae40 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f97a00740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555eced9ac70 con 0x555ecedbff10 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f97a00740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555ecedcf030 con 0x555eced7bec0 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f97a00740 10 client.0 _ll_register_callbacks cb 0x7f1f78046f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555ecee32cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555ecee32cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46714/0 (socket says 172.21.15.44:46714) 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f87fff640 1 -- 172.21.15.44:0/1642633434 learned_addr learned my addr 172.21.15.44:0/1642633434 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.143 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f8ed9d640 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbff10 0x555ecee33200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.140+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f87fff640 1 -- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 msgr2=0x555ecedc4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f87fff640 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f87fff640 1 -- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbff10 msgr2=0x555ecee33200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f87fff640 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbff10 0x555ecee33200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f87fff640 1 -- 172.21.15.44:0/1642633434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eced96590 con 0x555ecedbae40 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f87fff640 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555ecee32cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1f78083b80 tx=0x7f1f780abba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0df6e0c75e8f7bb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/1642633434 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7807b2a0 con 0x555ecedbae40 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/1642633434 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1f78077220 con 0x555ecedbae40 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f97a00740 1 -- 172.21.15.44:0/1642633434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f88024070 con 0x555ecedbae40 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/1642633434 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f1f780840b0 con 0x555ecedbae40 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f97a00740 10 client.4677 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:50.144 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f97a00740 1 -- 172.21.15.44:0/1642633434 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x555eced96590 con 0x555ecedbae40 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/1642633434 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f88024070 con 0x555ecedbae40 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/1642633434 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f1f78077420 con 0x555ecedbae40 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f97a00740 10 client.4677 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f97a00740 10 client.4677 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f97a00740 20 client.4677 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f97a00740 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x555ecedbe2b0 0x555ecedbe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f97a00740 4 client.4677 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f97a00740 1 -- 172.21.15.44:0/1642633434 --> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] -- command(tid 0: {"prefix": "status"}) -- 0x555ecece63a0 con 0x555ecedbe2b0 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.141+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x555ecedbe2b0 0x555ecedbe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f8e59c640 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x555ecedbe2b0 0x555ecedbe690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5498ef11b68cae94 server_cookie=9b605013b2d3ccb4 in_seq=0 out_seq=0 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f6d7fa640 10 client.4677 ms_handle_connect on v2:172.21.15.70:6833/4058009549 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f6d7fa640 1 -- 172.21.15.44:0/1642633434 <== mds.3 v2:172.21.15.70:6833/4058009549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x555ecece63a0 con 0x555ecedbe2b0 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f6d7fa640 10 client.4677 handle_command_reply: tid=0 2024-04-22T07:04:50.145 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 client.4677 shutdown 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 -- 172.21.15.44:0/1642633434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 msgr2=0x555ecee32cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555ecee32cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1f78083b80 tx=0x7f1f780abba0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 -- 172.21.15.44:0/1642633434 shutdown_connections 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x555ecedbe2b0 0x555ecedbe690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555ecedbff10 0x555ecee33200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f6d7fa640 0 client.4677 ms_handle_reset on v2:172.21.15.70:6833/4058009549 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555ecedbae40 0x555ecee32cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 --2- 172.21.15.44:0/1642633434 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555eced7bec0 0x555ecedc4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 -- 172.21.15.44:0/1642633434 >> 172.21.15.44:0/1642633434 conn(0x555eced77ca0 msgr2=0x555eced7aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 -- 172.21.15.44:0/1642633434 shutdown_connections 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 1 -- 172.21.15.44:0/1642633434 wait complete. 2024-04-22T07:04:50.146 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.142+0000 7f1f97a00740 20 client.4677 trim_cache size 0 max 16384 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "dc21e26c-6c65-477f-a367-ffb001b78b47", 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 3, 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4278, 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 15.276690026000001, 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 23, 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 23, 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 43.00069809 2024-04-22T07:04:50.147 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:04:50.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 -- 172.21.15.44:0/2137317796 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1f7006c920 msgr2=0x7f1f7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1f7006c920 0x7f1f7006ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000c00 tx=0x7f1f80001550 comp rx=0 tx=0).stop 2024-04-22T07:04:50.148 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 -- 172.21.15.44:0/2137317796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f90071d30 msgr2=0x7f1f900c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f90071d30 0x7f1f900c3f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1f78035040 tx=0x7f1f780046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 -- 172.21.15.44:0/2137317796 shutdown_connections 2024-04-22T07:04:50.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f1f7006c920 0x7f1f7006ede0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f9015add0 0x7f1f900c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1f90071d30 0x7f1f900c3f20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 --2- 172.21.15.44:0/2137317796 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1f90071950 0x7f1f90145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 -- 172.21.15.44:0/2137317796 >> 172.21.15.44:0/2137317796 conn(0x7f1f9006b970 msgr2=0x7f1f90152f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 -- 172.21.15.44:0/2137317796 shutdown_connections 2024-04-22T07:04:50.149 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.143+0000 7f1f94e11640 1 -- 172.21.15.44:0/2137317796 wait complete. 2024-04-22T07:04:50.151 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-22T07:04:50.252 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.255+0000 7f2f922e0640 1 Processor -- start 2024-04-22T07:04:50.253 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.255+0000 7f2f922e0640 1 -- start start 2024-04-22T07:04:50.254 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.255+0000 7f2f922e0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2f8c0cd6f0 0x7f2f8c15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.255+0000 7f2f922e0640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f8c15ee90 con 0x7f2f8c154f70 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.255+0000 7f2f922e0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f8c15f170 con 0x7f2f8c0cb3e0 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.255+0000 7f2f922e0640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2f8c0cd260 con 0x7f2f8c0cd6f0 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.256+0000 7f2f8b7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2f8c0cd6f0 0x7f2f8c15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.256+0000 7f2f8b7fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2f8c0cd6f0 0x7f2f8c15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46726/0 (socket says 172.21.15.44:46726) 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.256+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/3459790054 learned_addr learned my addr 172.21.15.44:0/3459790054 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.256+0000 7f2f8bfff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f2f8c154f70 0x7f2f8c0c5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:32986/0 (socket says 172.21.15.44:32986) 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.256+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/3459790054 >> v1:172.21.15.70:6789/0 conn(0x7f2f8c0cb3e0 legacy=0x7f2f8c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.256+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/3459790054 >> v1:172.21.15.44:6789/0 conn(0x7f2f8c154f70 legacy=0x7f2f8c0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.256+0000 7f2f8affd640 1 -- 172.21.15.44:0/3459790054 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2409674620 0 0) 0x7f2f8c15ee90 con 0x7f2f8c154f70 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.256+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/3459790054 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c0cc930 con 0x7f2f8c0cd6f0 2024-04-22T07:04:50.255 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/3459790054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cd6f0 0x7f2f8c15ad30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009870 tx=0x7f2f8002bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ba9111182d0988 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f8affd640 1 -- 172.21.15.44:0/3459790054 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8002d870 con 0x7f2f8c0cd6f0 2024-04-22T07:04:50.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f8affd640 1 -- 172.21.15.44:0/3459790054 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f8002da10 con 0x7f2f8c0cd6f0 2024-04-22T07:04:50.256 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f8affd640 1 -- 172.21.15.44:0/3459790054 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8002dde0 con 0x7f2f8c0cd6f0 2024-04-22T07:04:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f922e0640 1 -- 172.21.15.44:0/3459790054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cd6f0 msgr2=0x7f2f8c15ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f922e0640 1 --2- 172.21.15.44:0/3459790054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cd6f0 0x7f2f8c15ad30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009870 tx=0x7f2f8002bab0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f922e0640 1 -- 172.21.15.44:0/3459790054 shutdown_connections 2024-04-22T07:04:50.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f922e0640 1 --2- 172.21.15.44:0/3459790054 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cd6f0 0x7f2f8c15ad30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f922e0640 1 -- 172.21.15.44:0/3459790054 >> 172.21.15.44:0/3459790054 conn(0x7f2f8c067d40 msgr2=0x7f2f8c06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f922e0640 1 -- 172.21.15.44:0/3459790054 shutdown_connections 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.257+0000 7f2f922e0640 1 -- 172.21.15.44:0/3459790054 wait complete. 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.259+0000 7f2f922e0640 1 Processor -- start 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.259+0000 7f2f922e0640 1 -- start start 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.259+0000 7f2f922e0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cb3e0 0x7f2f8c15d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.259+0000 7f2f922e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8c0cd6f0 0x7f2f8c1e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.259+0000 7f2f922e0640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8c154f70 0x7f2f8c153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.259+0000 7f2f922e0640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c155350 con 0x7f2f8c0cb3e0 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.259+0000 7f2f922e0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c0ccef0 con 0x7f2f8c0cd6f0 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.259+0000 7f2f922e0640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2f8c1e6830 con 0x7f2f8c154f70 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cb3e0 0x7f2f8c15d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.259 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cb3e0 0x7f2f8c15d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46728/0 (socket says 172.21.15.44:46728) 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8bfff640 1 -- 172.21.15.44:0/1983820028 learned_addr learned my addr 172.21.15.44:0/1983820028 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8c0cd6f0 0x7f2f8c1e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f9087d640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8c154f70 0x7f2f8c153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8bfff640 1 -- 172.21.15.44:0/1983820028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8c154f70 msgr2=0x7f2f8c153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8c154f70 0x7f2f8c153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8bfff640 1 -- 172.21.15.44:0/1983820028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8c0cd6f0 msgr2=0x7f2f8c1e4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8c0cd6f0 0x7f2f8c1e4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8bfff640 1 -- 172.21.15.44:0/1983820028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c15de70 con 0x7f2f8c0cb3e0 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8c0cd6f0 0x7f2f8c1e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cb3e0 0x7f2f8c15d930 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00c4a0 tx=0x7f2f7c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f88ff9640 1 -- 172.21.15.44:0/1983820028 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c018070 con 0x7f2f8c0cb3e0 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f88ff9640 1 -- 172.21.15.44:0/1983820028 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c0074e0 con 0x7f2f8c0cb3e0 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f922e0640 1 -- 172.21.15.44:0/1983820028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c1e5110 con 0x7f2f8c0cb3e0 2024-04-22T07:04:50.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f88ff9640 1 -- 172.21.15.44:0/1983820028 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c004a20 con 0x7f2f8c0cb3e0 2024-04-22T07:04:50.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.260+0000 7f2f922e0640 1 -- 172.21.15.44:0/1983820028 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c1e40f0 con 0x7f2f8c0cb3e0 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.262+0000 7f2f88ff9640 1 -- 172.21.15.44:0/1983820028 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f7c01b070 con 0x7f2f8c0cb3e0 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.262+0000 7f2f88ff9640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f2f7806c8f0 0x7f2f7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.263+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f2f7806c8f0 0x7f2f7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.263+0000 7f2f88ff9640 1 -- 172.21.15.44:0/1983820028 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7c08ae20 con 0x7f2f8c0cb3e0 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.263+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f2f7806c8f0 0x7f2f7806edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2f80036000 tx=0x7f2f80039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.263+0000 7f2f94fab740 1 Processor -- start 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.263+0000 7f2f94fab740 1 -- start start 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 0x557454729f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546e1ec0 0x55745472a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546ddca0 0x557454705ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f94fab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x557454644b10 con 0x5574546e1ec0 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f94fab740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5574545e2240 con 0x5574546ddca0 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f94fab740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557454640ad0 con 0x557454720e40 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f8b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546e1ec0 0x55745472a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f8b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546e1ec0 0x55745472a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46730/0 (socket says 172.21.15.44:46730) 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/3700585680 learned_addr learned my addr 172.21.15.44:0/3700585680 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/3700585680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 0x557454729f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f9087d640 1 --2- 172.21.15.44:0/3700585680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546ddca0 0x557454705ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/3700585680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 msgr2=0x557454729f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/3700585680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 0x557454729f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/3700585680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546ddca0 msgr2=0x557454705ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/3700585680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546ddca0 0x557454705ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/3700585680 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574543572d0 con 0x5574546e1ec0 2024-04-22T07:04:50.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.264+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/3700585680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 0x557454729f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/3700585680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546e1ec0 0x55745472a4a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009630 tx=0x7f2f80030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61445b73989035e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f71ffb640 1 -- 172.21.15.44:0/3700585680 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80053070 con 0x5574546e1ec0 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f71ffb640 1 -- 172.21.15.44:0/3700585680 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f80037800 con 0x5574546e1ec0 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f71ffb640 1 -- 172.21.15.44:0/3700585680 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80037bb0 con 0x5574546e1ec0 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f94fab740 1 -- 172.21.15.44:0/3700585680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546e1ec0 msgr2=0x55745472a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f94fab740 1 --2- 172.21.15.44:0/3700585680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546e1ec0 0x55745472a4a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009630 tx=0x7f2f80030000 comp rx=0 tx=0).stop 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f94fab740 1 -- 172.21.15.44:0/3700585680 shutdown_connections 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f94fab740 1 --2- 172.21.15.44:0/3700585680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546ddca0 0x557454705ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f94fab740 1 --2- 172.21.15.44:0/3700585680 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546e1ec0 0x55745472a4a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f94fab740 1 --2- 172.21.15.44:0/3700585680 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 0x557454729f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f94fab740 1 -- 172.21.15.44:0/3700585680 >> 172.21.15.44:0/3700585680 conn(0x5574546dc990 msgr2=0x5574546df290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f94fab740 1 -- 172.21.15.44:0/3700585680 shutdown_connections 2024-04-22T07:04:50.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.265+0000 7f2f94fab740 1 -- 172.21.15.44:0/3700585680 wait complete. 2024-04-22T07:04:50.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.266+0000 7f2f94fab740 1 -- start start 2024-04-22T07:04:50.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.266+0000 7f2f94fab740 1 Processor -- start 2024-04-22T07:04:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546e1ec0 0x5574547283b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x5574546d9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546da460 0x55745471a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f94fab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5574546e0250 con 0x557454720e40 2024-04-22T07:04:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f94fab740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5574545cd190 con 0x5574546e1ec0 2024-04-22T07:04:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f94fab740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5574546e0530 con 0x5574546da460 2024-04-22T07:04:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f94fab740 10 client.0 _ll_register_callbacks cb 0x5574545c88b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f8b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x5574546d9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f8b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x5574546d9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46732/0 (socket says 172.21.15.44:46732) 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/2199677902 learned_addr learned my addr 172.21.15.44:0/2199677902 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f9087d640 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546da460 0x55745471a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546e1ec0 0x5574547283b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/2199677902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546da460 msgr2=0x55745471a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546da460 0x55745471a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/2199677902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546e1ec0 msgr2=0x5574547283b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546e1ec0 0x5574547283b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.267+0000 7f2f8b7fe640 1 -- 172.21.15.44:0/2199677902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574544216b0 con 0x557454720e40 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x5574546d9630 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009990 tx=0x7f2f80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555a19a3c72bb38e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f6affd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f6affd640 1 -- 172.21.15.44:0/2199677902 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80053030 con 0x557454720e40 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f6affd640 1 -- 172.21.15.44:0/2199677902 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2f80037600 con 0x557454720e40 2024-04-22T07:04:50.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f94fab740 1 -- 172.21.15.44:0/2199677902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f80037800 con 0x557454720e40 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f6affd640 1 -- 172.21.15.44:0/2199677902 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2f80061050 con 0x557454720e40 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f94fab740 10 client.4692 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f94fab740 1 -- 172.21.15.44:0/2199677902 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5574547288f0 con 0x557454720e40 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f6affd640 1 -- 172.21.15.44:0/2199677902 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f80037800 con 0x557454720e40 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f6affd640 1 -- 172.21.15.44:0/2199677902 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f2f80064020 con 0x557454720e40 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f94fab740 10 client.4692 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f94fab740 10 client.4692 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f94fab740 20 client.4692 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f94fab740 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x557454734f40 0x5574547407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f94fab740 4 client.4692 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f94fab740 1 -- 172.21.15.44:0/2199677902 --> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557454652530 con 0x557454734f40 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.268+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x557454734f40 0x5574547407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f6affd640 10 client.4692 ms_handle_connect on v2:172.21.15.44:6836/1520421043 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x557454734f40 0x5574547407e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=635b7bdf08d6999f server_cookie=c02b473db1fa65e2 in_seq=0 out_seq=0 2024-04-22T07:04:50.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f6affd640 1 -- 172.21.15.44:0/2199677902 <== mds.2 v2:172.21.15.44:6836/1520421043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557454652530 con 0x557454734f40 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f6affd640 10 client.4692 handle_command_reply: tid=0 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f94fab740 1 client.4692 shutdown 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f94fab740 1 -- 172.21.15.44:0/2199677902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 msgr2=0x5574546d9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f94fab740 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x5574546d9630 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009990 tx=0x7f2f80004730 comp rx=0 tx=0).stop 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f94fab740 1 -- 172.21.15.44:0/2199677902 shutdown_connections 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f94fab740 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x557454734f40 0x5574547407e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f94fab740 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546da460 0x55745471a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f94fab740 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x5574546d9630 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f94fab740 1 --2- 172.21.15.44:0/2199677902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546e1ec0 0x5574547283b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.269+0000 7f2f94fab740 1 -- 172.21.15.44:0/2199677902 >> 172.21.15.44:0/2199677902 conn(0x5574546ddca0 msgr2=0x5574546fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.270+0000 7f2f6affd640 0 client.4692 ms_handle_reset on v2:172.21.15.44:6836/1520421043 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.270+0000 7f2f94fab740 1 -- 172.21.15.44:0/2199677902 shutdown_connections 2024-04-22T07:04:50.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.270+0000 7f2f94fab740 1 -- 172.21.15.44:0/2199677902 wait complete. 2024-04-22T07:04:50.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.270+0000 7f2f94fab740 20 client.4692 trim_cache size 0 max 16384 2024-04-22T07:04:50.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.288+0000 7f2f94fab740 1 Processor -- start 2024-04-22T07:04:50.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.288+0000 7f2f94fab740 1 -- start start 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546da460 0x5574546f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546e1ec0 0x5574546f8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 0x557454724240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f94fab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1fcf80 con 0x5574546da460 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f94fab740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c14b7c0 con 0x5574546e1ec0 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f94fab740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557454727200 con 0x557454720e40 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546da460 0x5574546f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546da460 0x5574546f7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46736/0 (socket says 172.21.15.44:46736) 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f8bfff640 1 -- 172.21.15.44:0/396633133 learned_addr learned my addr 172.21.15.44:0/396633133 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/396633133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546e1ec0 0x5574546f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f9087d640 1 --2- 172.21.15.44:0/396633133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 0x557454724240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f8bfff640 1 -- 172.21.15.44:0/396633133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 msgr2=0x557454724240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/396633133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 0x557454724240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f8bfff640 1 -- 172.21.15.44:0/396633133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546e1ec0 msgr2=0x5574546f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/396633133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546e1ec0 0x5574546f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f8bfff640 1 -- 172.21.15.44:0/396633133 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574546dc1a0 con 0x5574546da460 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/396633133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546da460 0x5574546f7b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c058b70 tx=0x7f2f7c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614a97e34b4abca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f717fa640 1 -- 172.21.15.44:0/396633133 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c086020 con 0x5574546da460 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f717fa640 1 -- 172.21.15.44:0/396633133 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c056c80 con 0x5574546da460 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.289+0000 7f2f717fa640 1 -- 172.21.15.44:0/396633133 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c055250 con 0x5574546da460 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.290+0000 7f2f94fab740 1 -- 172.21.15.44:0/396633133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546da460 msgr2=0x5574546f7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.290+0000 7f2f94fab740 1 --2- 172.21.15.44:0/396633133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546da460 0x5574546f7b30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c058b70 tx=0x7f2f7c009040 comp rx=0 tx=0).stop 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.290+0000 7f2f94fab740 1 -- 172.21.15.44:0/396633133 shutdown_connections 2024-04-22T07:04:50.289 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.290+0000 7f2f94fab740 1 --2- 172.21.15.44:0/396633133 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x557454720e40 0x557454724240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.290+0000 7f2f94fab740 1 --2- 172.21.15.44:0/396633133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546e1ec0 0x5574546f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.290+0000 7f2f94fab740 1 --2- 172.21.15.44:0/396633133 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5574546da460 0x5574546f7b30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.290+0000 7f2f94fab740 1 -- 172.21.15.44:0/396633133 >> 172.21.15.44:0/396633133 conn(0x5574546ddca0 msgr2=0x557454720400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.290+0000 7f2f94fab740 1 -- 172.21.15.44:0/396633133 shutdown_connections 2024-04-22T07:04:50.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.290+0000 7f2f94fab740 1 -- 172.21.15.44:0/396633133 wait complete. 2024-04-22T07:04:50.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.290+0000 7f2f94fab740 1 -- start start 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.291+0000 7f2f94fab740 1 Processor -- start 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.291+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546da460 0x55745479ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.291+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546e1ec0 0x5574547451c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.291+0000 7f2f94fab740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x557454791b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.291+0000 7f2f94fab740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5574546d9bc0 con 0x557454720e40 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.291+0000 7f2f94fab740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5574547293f0 con 0x5574546da460 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.291+0000 7f2f94fab740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x557454734dc0 con 0x5574546e1ec0 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.291+0000 7f2f94fab740 10 client.0 _ll_register_callbacks cb 0x7f2f80051070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f9087d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x557454791b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546da460 0x55745479ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546da460 0x55745479ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.44:40948/0 (socket says 172.21.15.44:40948) 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8bfff640 1 -- 172.21.15.44:0/3597498880 learned_addr learned my addr 172.21.15.44:0/3597498880 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8b7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546e1ec0 0x5574547451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8bfff640 1 -- 172.21.15.44:0/3597498880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546e1ec0 msgr2=0x5574547451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546e1ec0 0x5574547451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8bfff640 1 -- 172.21.15.44:0/3597498880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 msgr2=0x557454791b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x557454791b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8bfff640 1 -- 172.21.15.44:0/3597498880 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557454745700 con 0x5574546da460 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f9087d640 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x557454791b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546e1ec0 0x5574547451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f8bfff640 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546da460 0x55745479ec80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c06f040 tx=0x7f2f7c00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b17c4cd9eac117 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f6affd640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f6affd640 1 -- 172.21.15.44:0/3597498880 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c086070 con 0x5574546da460 2024-04-22T07:04:50.295 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f6affd640 1 -- 172.21.15.44:0/3597498880 <== mon.1 v2:172.21.15.70:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2f7c004bc0 con 0x5574546da460 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f94fab740 1 -- 172.21.15.44:0/3597498880 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f7c056c80 con 0x5574546da460 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f6affd640 1 -- 172.21.15.44:0/3597498880 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f2f7c058be0 con 0x5574546da460 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f94fab740 10 client.4720 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f94fab740 1 -- 172.21.15.44:0/3597498880 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5574547044f0 con 0x5574546da460 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f6affd640 1 -- 172.21.15.44:0/3597498880 <== mon.1 v2:172.21.15.70:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f7c056c80 con 0x5574546da460 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.292+0000 7f2f6affd640 1 -- 172.21.15.44:0/3597498880 <== mon.1 v2:172.21.15.70:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f2f7c058030 con 0x5574546da460 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 10 client.4720 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 10 client.4720 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 20 client.4720 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x55745479b9d0 0x55745479bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 4 client.4720 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 -- 172.21.15.44:0/3597498880 --> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] -- command(tid 0: {"prefix": "status"}) -- 0x55745464c3a0 con 0x55745479b9d0 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x55745479b9d0 0x55745479bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f6affd640 10 client.4720 ms_handle_connect on v2:172.21.15.44:6836/1520421043 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f8b7fe640 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x55745479b9d0 0x55745479bdb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fccfad60d9c4967f server_cookie=2a4dcc3009fa4491 in_seq=0 out_seq=0 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f6affd640 1 -- 172.21.15.44:0/3597498880 <== mds.2 v2:172.21.15.44:6836/1520421043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55745464c3a0 con 0x55745479b9d0 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f6affd640 10 client.4720 handle_command_reply: tid=0 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 client.4720 shutdown 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 -- 172.21.15.44:0/3597498880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546da460 msgr2=0x55745479ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546da460 0x55745479ec80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c06f040 tx=0x7f2f7c00cf90 comp rx=0 tx=0).stop 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 -- 172.21.15.44:0/3597498880 shutdown_connections 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x55745479b9d0 0x55745479bdb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5574546e1ec0 0x5574547451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x557454720e40 0x557454791b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 --2- 172.21.15.44:0/3597498880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5574546da460 0x55745479ec80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f6affd640 0 client.4720 ms_handle_reset on v2:172.21.15.44:6836/1520421043 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 -- 172.21.15.44:0/3597498880 >> 172.21.15.44:0/3597498880 conn(0x5574546ddca0 msgr2=0x5574546e0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 -- 172.21.15.44:0/3597498880 shutdown_connections 2024-04-22T07:04:50.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.293+0000 7f2f94fab740 1 -- 172.21.15.44:0/3597498880 wait complete. 2024-04-22T07:04:50.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f94fab740 20 client.4720 trim_cache size 0 max 16384 2024-04-22T07:04:50.298 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:04:50.298 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "dc21e26c-6c65-477f-a367-ffb001b78b47", 2024-04-22T07:04:50.298 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2024-04-22T07:04:50.298 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4300, 2024-04-22T07:04:50.298 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T07:04:50.298 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T07:04:50.298 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T07:04:50.299 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 21.446844808000002, 2024-04-22T07:04:50.299 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:50.299 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 23, 2024-04-22T07:04:50.299 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 23, 2024-04-22T07:04:50.299 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 43.162881028999998 2024-04-22T07:04:50.299 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:04:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 -- 172.21.15.44:0/1983820028 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f2f7806c8f0 msgr2=0x7f2f7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f2f7806c8f0 0x7f2f7806edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2f80036000 tx=0x7f2f80039040 comp rx=0 tx=0).stop 2024-04-22T07:04:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 -- 172.21.15.44:0/1983820028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cb3e0 msgr2=0x7f2f8c15d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cb3e0 0x7f2f8c15d930 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00c4a0 tx=0x7f2f7c00c960 comp rx=0 tx=0).stop 2024-04-22T07:04:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 -- 172.21.15.44:0/1983820028 shutdown_connections 2024-04-22T07:04:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f2f7806c8f0 0x7f2f7806edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2f8c154f70 0x7f2f8c153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f8c0cd6f0 0x7f2f8c1e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 --2- 172.21.15.44:0/1983820028 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2f8c0cb3e0 0x7f2f8c15d930 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 -- 172.21.15.44:0/1983820028 >> 172.21.15.44:0/1983820028 conn(0x7f2f8c067d40 msgr2=0x7f2f8c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 -- 172.21.15.44:0/1983820028 shutdown_connections 2024-04-22T07:04:50.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.294+0000 7f2f922e0640 1 -- 172.21.15.44:0/1983820028 wait complete. 2024-04-22T07:04:50.303 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T07:04:50.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.406+0000 7fae40ceb640 1 Processor -- start 2024-04-22T07:04:50.404 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.406+0000 7fae40ceb640 1 -- start start 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae40ceb640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c155120 0x7fae3c0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae40ceb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae3c0caec0 0x7fae3c0d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae40ceb640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae3c155500 con 0x7fae3c0d0f10 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae40ceb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fae3c072f90 con 0x7fae3c0caec0 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae40ceb640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fae3c15b0d0 con 0x7fae3c155120 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae3ad9d640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fae3c0d0f10 0x7fae3c15e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:32988/0 (socket says 172.21.15.44:32988) 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae3ad9d640 1 -- 172.21.15.44:0/392635025 learned_addr learned my addr 172.21.15.44:0/392635025 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae3a59c640 1 --2- 172.21.15.44:0/392635025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c155120 0x7fae3c0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae33fff640 1 --2- 172.21.15.44:0/392635025 >> v2:172.21.15.70:3300/0 conn(0x7fae3c0caec0 0x7fae3c0d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae39d9b640 1 -- 172.21.15.44:0/392635025 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 213746593 0 0) 0x7fae3c155500 con 0x7fae3c0d0f10 2024-04-22T07:04:50.405 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae3a59c640 1 -- 172.21.15.44:0/392635025 >> v2:172.21.15.70:3300/0 conn(0x7fae3c0caec0 msgr2=0x7fae3c0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae3a59c640 1 --2- 172.21.15.44:0/392635025 >> v2:172.21.15.70:3300/0 conn(0x7fae3c0caec0 0x7fae3c0d09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae3a59c640 1 -- 172.21.15.44:0/392635025 >> v1:172.21.15.44:6789/0 conn(0x7fae3c0d0f10 legacy=0x7fae3c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.406 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.407+0000 7fae3a59c640 1 -- 172.21.15.44:0/392635025 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3c15b3b0 con 0x7fae3c155120 2024-04-22T07:04:50.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.408+0000 7fae3a59c640 1 --2- 172.21.15.44:0/392635025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c155120 0x7fae3c0c6380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0097a0 tx=0x7fae2c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8f954db6e829b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.408+0000 7fae39d9b640 1 -- 172.21.15.44:0/392635025 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c035020 con 0x7fae3c155120 2024-04-22T07:04:50.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.408+0000 7fae39d9b640 1 -- 172.21.15.44:0/392635025 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2c031830 con 0x7fae3c155120 2024-04-22T07:04:50.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.408+0000 7fae39d9b640 1 -- 172.21.15.44:0/392635025 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c031c00 con 0x7fae3c155120 2024-04-22T07:04:50.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.409+0000 7fae40ceb640 1 -- 172.21.15.44:0/392635025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c155120 msgr2=0x7fae3c0c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.409+0000 7fae40ceb640 1 --2- 172.21.15.44:0/392635025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c155120 0x7fae3c0c6380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0097a0 tx=0x7fae2c02b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.409+0000 7fae40ceb640 1 -- 172.21.15.44:0/392635025 shutdown_connections 2024-04-22T07:04:50.407 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.409+0000 7fae40ceb640 1 --2- 172.21.15.44:0/392635025 >> v2:172.21.15.70:3300/0 conn(0x7fae3c0caec0 0x7fae3c0d09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.409+0000 7fae40ceb640 1 --2- 172.21.15.44:0/392635025 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c155120 0x7fae3c0c6380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.409+0000 7fae40ceb640 1 -- 172.21.15.44:0/392635025 >> 172.21.15.44:0/392635025 conn(0x7fae3c0676b0 msgr2=0x7fae3c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.409+0000 7fae40ceb640 1 -- 172.21.15.44:0/392635025 shutdown_connections 2024-04-22T07:04:50.408 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.409+0000 7fae40ceb640 1 -- 172.21.15.44:0/392635025 wait complete. 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.410+0000 7fae40ceb640 1 Processor -- start 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.410+0000 7fae40ceb640 1 -- start start 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae40ceb640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c0caec0 0x7fae3c14b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae40ceb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae3c0d0f10 0x7fae3c1f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae40ceb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae3c155120 0x7fae3c1f0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae40ceb640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c15bc50 con 0x7fae3c0d0f10 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae40ceb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c15b970 con 0x7fae3c155120 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae40ceb640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fae3c15e390 con 0x7fae3c0caec0 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae33fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae3c0d0f10 0x7fae3c1f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae33fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae3c0d0f10 0x7fae3c1f0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46754/0 (socket says 172.21.15.44:46754) 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae33fff640 1 -- 172.21.15.44:0/2250006154 learned_addr learned my addr 172.21.15.44:0/2250006154 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.410 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae3a59c640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c0caec0 0x7fae3c14b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae3ad9d640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae3c155120 0x7fae3c1f0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae33fff640 1 -- 172.21.15.44:0/2250006154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c0caec0 msgr2=0x7fae3c14b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae33fff640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c0caec0 0x7fae3c14b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae33fff640 1 -- 172.21.15.44:0/2250006154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae3c155120 msgr2=0x7fae3c1f0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae33fff640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae3c155120 0x7fae3c1f0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae33fff640 1 -- 172.21.15.44:0/2250006154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3c1f3d60 con 0x7fae3c0d0f10 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae33fff640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae3c0d0f10 0x7fae3c1f0a40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fae2400e9b0 tx=0x7fae2400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae32ffd640 1 -- 172.21.15.44:0/2250006154 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24017070 con 0x7fae3c0d0f10 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae32ffd640 1 -- 172.21.15.44:0/2250006154 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2400c9a0 con 0x7fae3c0d0f10 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae40ceb640 1 -- 172.21.15.44:0/2250006154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3c1f4040 con 0x7fae3c0d0f10 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae32ffd640 1 -- 172.21.15.44:0/2250006154 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2400cd90 con 0x7fae3c0d0f10 2024-04-22T07:04:50.411 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.411+0000 7fae40ceb640 1 -- 172.21.15.44:0/2250006154 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3c1f4600 con 0x7fae3c0d0f10 2024-04-22T07:04:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.413+0000 7fae32ffd640 1 -- 172.21.15.44:0/2250006154 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae2401a070 con 0x7fae3c0d0f10 2024-04-22T07:04:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.413+0000 7fae32ffd640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fae1c06c8f0 0x7fae1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.413+0000 7fae32ffd640 1 -- 172.21.15.44:0/2250006154 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fae2408bed0 con 0x7fae3c0d0f10 2024-04-22T07:04:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.413+0000 7fae3a59c640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fae1c06c8f0 0x7fae1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.414+0000 7fae3a59c640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fae1c06c8f0 0x7fae1c06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae2c036000 tx=0x7fae2c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.414+0000 7fae439bc740 1 Processor -- start 2024-04-22T07:04:50.414 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.414+0000 7fae439bc740 1 -- start start 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae439bc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a82ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae439bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a8304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae439bc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a7e3ca0 0x56419a80bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae439bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56419a74ab10 con 0x56419a7e7ec0 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae439bc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56419a6e8240 con 0x56419a7e3ca0 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae439bc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56419a746ad0 con 0x56419a826e40 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae33fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a8304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae33fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a8304a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46758/0 (socket says 172.21.15.44:46758) 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae33fff640 1 -- 172.21.15.44:0/1842784788 learned_addr learned my addr 172.21.15.44:0/1842784788 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae3a59c640 1 --2- 172.21.15.44:0/1842784788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a82ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae3ad9d640 1 --2- 172.21.15.44:0/1842784788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a7e3ca0 0x56419a80bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae33fff640 1 -- 172.21.15.44:0/1842784788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 msgr2=0x56419a82ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae33fff640 1 --2- 172.21.15.44:0/1842784788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a82ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.415 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae33fff640 1 -- 172.21.15.44:0/1842784788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a7e3ca0 msgr2=0x56419a80bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae33fff640 1 --2- 172.21.15.44:0/1842784788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a7e3ca0 0x56419a80bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae33fff640 1 -- 172.21.15.44:0/1842784788 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419a45d2d0 con 0x56419a7e7ec0 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.415+0000 7fae33fff640 1 --2- 172.21.15.44:0/1842784788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a8304a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c520 tx=0x7fae240596a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857248ff8e662cc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae1bfff640 1 -- 172.21.15.44:0/1842784788 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24021040 con 0x56419a7e7ec0 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae1bfff640 1 -- 172.21.15.44:0/1842784788 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae24057f10 con 0x56419a7e7ec0 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae1bfff640 1 -- 172.21.15.44:0/1842784788 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae24059060 con 0x56419a7e7ec0 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae439bc740 1 -- 172.21.15.44:0/1842784788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 msgr2=0x56419a8304a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae439bc740 1 --2- 172.21.15.44:0/1842784788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a8304a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c520 tx=0x7fae240596a0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae439bc740 1 -- 172.21.15.44:0/1842784788 shutdown_connections 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae439bc740 1 --2- 172.21.15.44:0/1842784788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a7e3ca0 0x56419a80bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae439bc740 1 --2- 172.21.15.44:0/1842784788 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a8304a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae439bc740 1 --2- 172.21.15.44:0/1842784788 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a82ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae439bc740 1 -- 172.21.15.44:0/1842784788 >> 172.21.15.44:0/1842784788 conn(0x56419a7e2990 msgr2=0x56419a7e5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae439bc740 1 -- 172.21.15.44:0/1842784788 shutdown_connections 2024-04-22T07:04:50.416 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.416+0000 7fae439bc740 1 -- 172.21.15.44:0/1842784788 wait complete. 2024-04-22T07:04:50.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.417+0000 7fae439bc740 1 -- start start 2024-04-22T07:04:50.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.417+0000 7fae439bc740 1 Processor -- start 2024-04-22T07:04:50.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae439bc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a7e7ec0 0x56419a82a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.420 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae439bc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a826e40 0x56419a83b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae439bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a828180 0x56419a820300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae439bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56419a7e6250 con 0x56419a828180 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae439bc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56419a6d3190 con 0x56419a826e40 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae439bc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56419a7e6530 con 0x56419a7e7ec0 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae439bc740 10 client.0 _ll_register_callbacks cb 0x56419a6ce8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae3ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a828180 0x56419a820300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae3ad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a828180 0x56419a820300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46770/0 (socket says 172.21.15.44:46770) 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae3ad9d640 1 -- 172.21.15.44:0/4212579852 learned_addr learned my addr 172.21.15.44:0/4212579852 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae3a59c640 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a7e7ec0 0x56419a82a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae33fff640 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a826e40 0x56419a83b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae3ad9d640 1 -- 172.21.15.44:0/4212579852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a7e7ec0 msgr2=0x56419a82a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae3ad9d640 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a7e7ec0 0x56419a82a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae3ad9d640 1 -- 172.21.15.44:0/4212579852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a826e40 msgr2=0x56419a83b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.421 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae3ad9d640 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a826e40 0x56419a83b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae3ad9d640 1 -- 172.21.15.44:0/4212579852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56419a5276b0 con 0x56419a828180 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.418+0000 7fae3ad9d640 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a828180 0x56419a820300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fae34007310 tx=0x7fae340077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffdfd2a57f8a250 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae197fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae197fa640 1 -- 172.21.15.44:0/4212579852 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34007900 con 0x56419a828180 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae197fa640 1 -- 172.21.15.44:0/4212579852 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fae34007aa0 con 0x56419a828180 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae439bc740 1 -- 172.21.15.44:0/4212579852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae24057f10 con 0x56419a828180 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae197fa640 1 -- 172.21.15.44:0/4212579852 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fae34022050 con 0x56419a828180 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae439bc740 10 client.4710 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae439bc740 1 -- 172.21.15.44:0/4212579852 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x56419a82e6a0 con 0x56419a828180 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae197fa640 1 -- 172.21.15.44:0/4212579852 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae24057f10 con 0x56419a828180 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae197fa640 1 -- 172.21.15.44:0/4212579852 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fae34025020 con 0x56419a828180 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae439bc740 10 client.4710 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:50.422 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae439bc740 10 client.4710 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae439bc740 20 client.4710 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae439bc740 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x56419a7dfeb0 0x56419a846be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae439bc740 4 client.4710 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae439bc740 1 -- 172.21.15.44:0/4212579852 --> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56419a758530 con 0x56419a7dfeb0 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.419+0000 7fae3a59c640 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x56419a7dfeb0 0x56419a846be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae3a59c640 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x56419a7dfeb0 0x56419a846be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5abaa477bc216a3 server_cookie=274ee34cbb1e853b in_seq=0 out_seq=0 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae197fa640 10 client.4710 ms_handle_connect on v2:172.21.15.44:6837/1911557586 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae197fa640 1 -- 172.21.15.44:0/4212579852 <== mds.1 v2:172.21.15.44:6837/1911557586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56419a758530 con 0x56419a7dfeb0 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae197fa640 10 client.4710 handle_command_reply: tid=0 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae439bc740 1 client.4710 shutdown 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae439bc740 1 -- 172.21.15.44:0/4212579852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a828180 msgr2=0x56419a820300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae439bc740 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a828180 0x56419a820300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fae34007310 tx=0x7fae340077d0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae439bc740 1 -- 172.21.15.44:0/4212579852 shutdown_connections 2024-04-22T07:04:50.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae439bc740 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x56419a7dfeb0 0x56419a846be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae439bc740 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a828180 0x56419a820300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae439bc740 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a826e40 0x56419a83b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae197fa640 0 client.4710 ms_handle_reset on v2:172.21.15.44:6837/1911557586 2024-04-22T07:04:50.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.420+0000 7fae439bc740 1 --2- 172.21.15.44:0/4212579852 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a7e7ec0 0x56419a82a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.421+0000 7fae439bc740 1 -- 172.21.15.44:0/4212579852 >> 172.21.15.44:0/4212579852 conn(0x56419a7e3ca0 msgr2=0x56419a825bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.421+0000 7fae439bc740 1 -- 172.21.15.44:0/4212579852 shutdown_connections 2024-04-22T07:04:50.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.421+0000 7fae439bc740 1 -- 172.21.15.44:0/4212579852 wait complete. 2024-04-22T07:04:50.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.421+0000 7fae439bc740 20 client.4710 trim_cache size 0 max 16384 2024-04-22T07:04:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.439+0000 7fae439bc740 1 Processor -- start 2024-04-22T07:04:50.437 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.439+0000 7fae439bc740 1 -- start start 2024-04-22T07:04:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae439bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a7e5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae439bc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a82d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae439bc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a828180 0x56419a82dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.439 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae439bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c0cf0c0 con 0x56419a7e7ec0 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae439bc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56419a82e9e0 con 0x56419a828180 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae439bc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56419a7e1150 con 0x56419a826e40 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae3a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a7e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae3a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a7e5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46782/0 (socket says 172.21.15.44:46782) 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae3a59c640 1 -- 172.21.15.44:0/687933723 learned_addr learned my addr 172.21.15.44:0/687933723 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae3a59c640 1 -- 172.21.15.44:0/687933723 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 msgr2=0x56419a82d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae33fff640 1 --2- 172.21.15.44:0/687933723 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a82d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae3ad9d640 1 --2- 172.21.15.44:0/687933723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a828180 0x56419a82dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae3a59c640 1 --2- 172.21.15.44:0/687933723 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a82d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae3a59c640 1 -- 172.21.15.44:0/687933723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a828180 msgr2=0x56419a82dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae3a59c640 1 --2- 172.21.15.44:0/687933723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a828180 0x56419a82dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae3a59c640 1 -- 172.21.15.44:0/687933723 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56419a7df310 con 0x56419a7e7ec0 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae33fff640 1 --2- 172.21.15.44:0/687933723 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a82d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:50.440 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.440+0000 7fae3a59c640 1 --2- 172.21.15.44:0/687933723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a7e5740 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae2c02b910 tx=0x7fae2c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0c333919bf029c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae1b7fe640 1 -- 172.21.15.44:0/687933723 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c05c070 con 0x56419a7e7ec0 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae1b7fe640 1 -- 172.21.15.44:0/687933723 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2c031d90 con 0x56419a7e7ec0 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae1b7fe640 1 -- 172.21.15.44:0/687933723 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c035040 con 0x56419a7e7ec0 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae439bc740 1 -- 172.21.15.44:0/687933723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 msgr2=0x56419a7e5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae439bc740 1 --2- 172.21.15.44:0/687933723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a7e5740 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae2c02b910 tx=0x7fae2c030000 comp rx=0 tx=0).stop 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae439bc740 1 -- 172.21.15.44:0/687933723 shutdown_connections 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae439bc740 1 --2- 172.21.15.44:0/687933723 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a82d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae439bc740 1 --2- 172.21.15.44:0/687933723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a828180 0x56419a82dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae439bc740 1 --2- 172.21.15.44:0/687933723 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a7e5740 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae439bc740 1 -- 172.21.15.44:0/687933723 >> 172.21.15.44:0/687933723 conn(0x56419a7e3ca0 msgr2=0x56419a7e12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae439bc740 1 -- 172.21.15.44:0/687933723 shutdown_connections 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.441+0000 7fae439bc740 1 -- 172.21.15.44:0/687933723 wait complete. 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.442+0000 7fae439bc740 1 -- start start 2024-04-22T07:04:50.441 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.442+0000 7fae439bc740 1 Processor -- start 2024-04-22T07:04:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae439bc740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a82b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae439bc740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a81cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae439bc740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a828180 0x56419a8a6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae439bc740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x56419a83b9c0 con 0x56419a7e7ec0 2024-04-22T07:04:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae439bc740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x56419a824890 con 0x56419a828180 2024-04-22T07:04:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae439bc740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x56419a7e0370 con 0x56419a826e40 2024-04-22T07:04:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae439bc740 10 client.0 _ll_register_callbacks cb 0x7fae34004c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae3a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a82b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.445 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae3a59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a82b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46794/0 (socket says 172.21.15.44:46794) 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae3a59c640 1 -- 172.21.15.44:0/3860544427 learned_addr learned my addr 172.21.15.44:0/3860544427 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae3ad9d640 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a828180 0x56419a8a6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae33fff640 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a81cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae3a59c640 1 -- 172.21.15.44:0/3860544427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 msgr2=0x56419a81cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae3a59c640 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a81cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae3a59c640 1 -- 172.21.15.44:0/3860544427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a828180 msgr2=0x56419a8a6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae3a59c640 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a828180 0x56419a8a6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae3a59c640 1 -- 172.21.15.44:0/3860544427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56419a7e3720 con 0x56419a7e7ec0 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae3a59c640 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a82b540 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098c0 tx=0x7fae2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ca1b72da7a2a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae197fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae197fa640 1 -- 172.21.15.44:0/3860544427 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c05c030 con 0x56419a7e7ec0 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae197fa640 1 -- 172.21.15.44:0/3860544427 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fae2c0039e0 con 0x56419a7e7ec0 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae439bc740 1 -- 172.21.15.44:0/3860544427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae2c031d90 con 0x56419a7e7ec0 2024-04-22T07:04:50.446 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae197fa640 1 -- 172.21.15.44:0/3860544427 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fae2c035040 con 0x56419a7e7ec0 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae439bc740 10 client.4716 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.443+0000 7fae439bc740 1 -- 172.21.15.44:0/3860544427 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x56419a84c180 con 0x56419a7e7ec0 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae197fa640 1 -- 172.21.15.44:0/3860544427 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2c031d90 con 0x56419a7e7ec0 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae197fa640 1 -- 172.21.15.44:0/3860544427 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fae2c003be0 con 0x56419a7e7ec0 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae439bc740 10 client.4716 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae439bc740 10 client.4716 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae439bc740 20 client.4716 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae439bc740 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x56419a89f950 0x56419a89fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae439bc740 4 client.4716 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae439bc740 1 -- 172.21.15.44:0/3860544427 --> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] -- command(tid 0: {"prefix": "status"}) -- 0x56419a7523a0 con 0x56419a89f950 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae33fff640 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x56419a89f950 0x56419a89fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae197fa640 10 client.4716 ms_handle_connect on v2:172.21.15.44:6837/1911557586 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae33fff640 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x56419a89f950 0x56419a89fd30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52cffa115d16991a server_cookie=3cb0d108061445dd in_seq=0 out_seq=0 2024-04-22T07:04:50.447 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae197fa640 1 -- 172.21.15.44:0/3860544427 <== mds.1 v2:172.21.15.44:6837/1911557586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56419a7523a0 con 0x56419a89f950 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae197fa640 10 client.4716 handle_command_reply: tid=0 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae439bc740 1 client.4716 shutdown 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae439bc740 1 -- 172.21.15.44:0/3860544427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 msgr2=0x56419a82b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.444+0000 7fae439bc740 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a82b540 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae2c0098c0 tx=0x7fae2c004730 comp rx=0 tx=0).stop 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae439bc740 1 -- 172.21.15.44:0/3860544427 shutdown_connections 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae439bc740 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x56419a89f950 0x56419a89fd30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae439bc740 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x56419a826e40 0x56419a81cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae197fa640 0 client.4716 ms_handle_reset on v2:172.21.15.44:6837/1911557586 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae439bc740 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56419a828180 0x56419a8a6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae439bc740 1 --2- 172.21.15.44:0/3860544427 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x56419a7e7ec0 0x56419a82b540 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae439bc740 1 -- 172.21.15.44:0/3860544427 >> 172.21.15.44:0/3860544427 conn(0x56419a7e3ca0 msgr2=0x56419a825990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae439bc740 1 -- 172.21.15.44:0/3860544427 shutdown_connections 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae439bc740 1 -- 172.21.15.44:0/3860544427 wait complete. 2024-04-22T07:04:50.448 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae439bc740 20 client.4716 trim_cache size 0 max 16384 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "dc21e26c-6c65-477f-a367-ffb001b78b47", 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4303, 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 27.621699303, 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 23, 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 23, 2024-04-22T07:04:50.449 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 43.313598398000003 2024-04-22T07:04:50.450 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:04:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 -- 172.21.15.44:0/2250006154 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fae1c06c8f0 msgr2=0x7fae1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fae1c06c8f0 0x7fae1c06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae2c036000 tx=0x7fae2c008030 comp rx=0 tx=0).stop 2024-04-22T07:04:50.450 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 -- 172.21.15.44:0/2250006154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae3c0d0f10 msgr2=0x7fae3c1f0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae3c0d0f10 0x7fae3c1f0a40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fae2400e9b0 tx=0x7fae2400ee70 comp rx=0 tx=0).stop 2024-04-22T07:04:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 -- 172.21.15.44:0/2250006154 shutdown_connections 2024-04-22T07:04:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fae1c06c8f0 0x7fae1c06edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae3c155120 0x7fae3c1f0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fae3c0d0f10 0x7fae3c1f0a40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 --2- 172.21.15.44:0/2250006154 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fae3c0caec0 0x7fae3c14b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 -- 172.21.15.44:0/2250006154 >> 172.21.15.44:0/2250006154 conn(0x7fae3c0676b0 msgr2=0x7fae3c0b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 -- 172.21.15.44:0/2250006154 shutdown_connections 2024-04-22T07:04:50.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.445+0000 7fae40ceb640 1 -- 172.21.15.44:0/2250006154 wait complete. 2024-04-22T07:04:50.454 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-22T07:04:50.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.557+0000 7fa5c4a9a640 1 Processor -- start 2024-04-22T07:04:50.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.557+0000 7fa5c4a9a640 1 -- start start 2024-04-22T07:04:50.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5c4a9a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa5c0154f70 0x7fa5c015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5c4a9a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5c00cb3e0 0x7fa5c00cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5c4a9a640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5c015ee90 con 0x7fa5c00cd6f0 2024-04-22T07:04:50.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5c4a9a640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c00cd260 con 0x7fa5c0154f70 2024-04-22T07:04:50.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5c4a9a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fa5c015f170 con 0x7fa5c00cb3e0 2024-04-22T07:04:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5be59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa5c0154f70 0x7fa5c015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5be59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fa5c0154f70 0x7fa5c015ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46798/0 (socket says 172.21.15.44:46798) 2024-04-22T07:04:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5be59c640 1 -- 172.21.15.44:0/1485124228 learned_addr learned my addr 172.21.15.44:0/1485124228 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5bdd9b640 1 --1- 172.21.15.44:0/1485124228 >> v1:172.21.15.44:6789/0 conn(0x7fa5c00cd6f0 0x7fa5c00c5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:32992/0 (socket says 172.21.15.44:32992) 2024-04-22T07:04:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/1485124228 >> v2:172.21.15.70:3300/0 conn(0x7fa5c00cb3e0 0x7fa5c00cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5be59c640 1 -- 172.21.15.44:0/1485124228 >> v2:172.21.15.70:3300/0 conn(0x7fa5c00cb3e0 msgr2=0x7fa5c00cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5be59c640 1 --2- 172.21.15.44:0/1485124228 >> v2:172.21.15.70:3300/0 conn(0x7fa5c00cb3e0 0x7fa5c00cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5be59c640 1 -- 172.21.15.44:0/1485124228 >> v1:172.21.15.44:6789/0 conn(0x7fa5c00cd6f0 legacy=0x7fa5c00c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5bd59a640 1 -- 172.21.15.44:0/1485124228 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383461131 0 0) 0x7fa5c015ee90 con 0x7fa5c00cd6f0 2024-04-22T07:04:50.557 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.558+0000 7fa5be59c640 1 -- 172.21.15.44:0/1485124228 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c00cc930 con 0x7fa5c0154f70 2024-04-22T07:04:50.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.559+0000 7fa5be59c640 1 --2- 172.21.15.44:0/1485124228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c0154f70 0x7fa5c015ad30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00633d0 tx=0x7fa5b802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca19a33c09f44323 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.559+0000 7fa5bd59a640 1 -- 172.21.15.44:0/1485124228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b802f850 con 0x7fa5c0154f70 2024-04-22T07:04:50.558 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.559+0000 7fa5bd59a640 1 -- 172.21.15.44:0/1485124228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b802f9f0 con 0x7fa5c0154f70 2024-04-22T07:04:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.559+0000 7fa5bd59a640 1 -- 172.21.15.44:0/1485124228 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b802fdc0 con 0x7fa5c0154f70 2024-04-22T07:04:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.560+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/1485124228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c0154f70 msgr2=0x7fa5c015ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.560+0000 7fa5c4a9a640 1 --2- 172.21.15.44:0/1485124228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c0154f70 0x7fa5c015ad30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa5c00633d0 tx=0x7fa5b802ba30 comp rx=0 tx=0).stop 2024-04-22T07:04:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.560+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/1485124228 shutdown_connections 2024-04-22T07:04:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.560+0000 7fa5c4a9a640 1 --2- 172.21.15.44:0/1485124228 >> v2:172.21.15.70:3300/0 conn(0x7fa5c00cb3e0 0x7fa5c00cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.560+0000 7fa5c4a9a640 1 --2- 172.21.15.44:0/1485124228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c0154f70 0x7fa5c015ad30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.560+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/1485124228 >> 172.21.15.44:0/1485124228 conn(0x7fa5c0067d40 msgr2=0x7fa5c006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.560+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/1485124228 shutdown_connections 2024-04-22T07:04:50.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.560+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/1485124228 wait complete. 2024-04-22T07:04:50.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.561+0000 7fa5c4a9a640 1 Processor -- start 2024-04-22T07:04:50.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.561+0000 7fa5c4a9a640 1 -- start start 2024-04-22T07:04:50.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5c4a9a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00cb3e0 0x7fa5c015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5c4a9a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c00cd6f0 0x7fa5c0159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5c4a9a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5c0154f70 0x7fa5c0159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5c4a9a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c0155350 con 0x7fa5c00cb3e0 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5c4a9a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c00ccef0 con 0x7fa5c00cd6f0 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5c4a9a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c014f4b0 con 0x7fa5c0154f70 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5be59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00cb3e0 0x7fa5c015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5be59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00cb3e0 0x7fa5c015a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46802/0 (socket says 172.21.15.44:46802) 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5be59c640 1 -- 172.21.15.44:0/614994968 learned_addr learned my addr 172.21.15.44:0/614994968 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5bdd9b640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c00cd6f0 0x7fa5c0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5c0154f70 0x7fa5c0159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.562+0000 7fa5be59c640 1 -- 172.21.15.44:0/614994968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5c0154f70 msgr2=0x7fa5c0159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5be59c640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5c0154f70 0x7fa5c0159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5be59c640 1 -- 172.21.15.44:0/614994968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c00cd6f0 msgr2=0x7fa5c0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5be59c640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c00cd6f0 0x7fa5c0159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5be59c640 1 -- 172.21.15.44:0/614994968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c015acf0 con 0x7fa5c00cb3e0 2024-04-22T07:04:50.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5bdd9b640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c00cd6f0 0x7fa5c0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:50.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5be59c640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00cb3e0 0x7fa5c015a390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8000c00 tx=0x7fa5b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5aeffd640 1 -- 172.21.15.44:0/614994968 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8047070 con 0x7fa5c00cb3e0 2024-04-22T07:04:50.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5aeffd640 1 -- 172.21.15.44:0/614994968 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b8037d50 con 0x7fa5c00cb3e0 2024-04-22T07:04:50.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/614994968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c015aa10 con 0x7fa5c00cb3e0 2024-04-22T07:04:50.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5aeffd640 1 -- 172.21.15.44:0/614994968 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8004270 con 0x7fa5c00cb3e0 2024-04-22T07:04:50.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.563+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/614994968 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c01f1ad0 con 0x7fa5c00cb3e0 2024-04-22T07:04:50.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.564+0000 7fa5aeffd640 1 -- 172.21.15.44:0/614994968 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5b804a070 con 0x7fa5c00cb3e0 2024-04-22T07:04:50.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.565+0000 7fa5aeffd640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa5a006c920 0x7fa5a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.565+0000 7fa5aeffd640 1 -- 172.21.15.44:0/614994968 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa5b80add70 con 0x7fa5c00cb3e0 2024-04-22T07:04:50.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.565+0000 7fa5bdd9b640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa5a006c920 0x7fa5a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.565+0000 7fa5bdd9b640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa5a006c920 0x7fa5a006ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa5c015f030 tx=0x7fa5b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.566+0000 7fa5c7769740 1 Processor -- start 2024-04-22T07:04:50.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.566+0000 7fa5c7769740 1 -- start start 2024-04-22T07:04:50.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863b2ec0 0x5595863fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863aeca0 0x5595863d6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5c7769740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559586315b10 con 0x5595863f1e40 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5c7769740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5595862b3240 con 0x5595863b2ec0 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5c7769740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x559586311ad0 con 0x5595863aeca0 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5be59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5be59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863faf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46810/0 (socket says 172.21.15.44:46810) 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5be59c640 1 -- 172.21.15.44:0/4057520676 learned_addr learned my addr 172.21.15.44:0/4057520676 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/4057520676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863aeca0 0x5595863d6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5bdd9b640 1 --2- 172.21.15.44:0/4057520676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863b2ec0 0x5595863fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5be59c640 1 -- 172.21.15.44:0/4057520676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863aeca0 msgr2=0x5595863d6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5be59c640 1 --2- 172.21.15.44:0/4057520676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863aeca0 0x5595863d6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.567 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5be59c640 1 -- 172.21.15.44:0/4057520676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863b2ec0 msgr2=0x5595863fb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5be59c640 1 --2- 172.21.15.44:0/4057520676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863b2ec0 0x5595863fb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5be59c640 1 -- 172.21.15.44:0/4057520676 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595860282d0 con 0x5595863f1e40 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa5be59c640 1 --2- 172.21.15.44:0/4057520676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863faf60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8084150 tx=0x7fa5b807c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40d08a2b732adff server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.567+0000 7fa59ffff640 1 -- 172.21.15.44:0/4057520676 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b807fe50 con 0x5595863f1e40 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa59ffff640 1 -- 172.21.15.44:0/4057520676 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b8084680 con 0x5595863f1e40 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa59ffff640 1 -- 172.21.15.44:0/4057520676 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8079da0 con 0x5595863f1e40 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa5c7769740 1 -- 172.21.15.44:0/4057520676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 msgr2=0x5595863faf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa5c7769740 1 --2- 172.21.15.44:0/4057520676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863faf60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8084150 tx=0x7fa5b807c1a0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa5c7769740 1 -- 172.21.15.44:0/4057520676 shutdown_connections 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa5c7769740 1 --2- 172.21.15.44:0/4057520676 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863aeca0 0x5595863d6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa5c7769740 1 --2- 172.21.15.44:0/4057520676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863b2ec0 0x5595863fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa5c7769740 1 --2- 172.21.15.44:0/4057520676 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863faf60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa5c7769740 1 -- 172.21.15.44:0/4057520676 >> 172.21.15.44:0/4057520676 conn(0x5595863ad990 msgr2=0x5595863b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa5c7769740 1 -- 172.21.15.44:0/4057520676 shutdown_connections 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.568+0000 7fa5c7769740 1 -- 172.21.15.44:0/4057520676 wait complete. 2024-04-22T07:04:50.568 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.569+0000 7fa5c7769740 1 -- start start 2024-04-22T07:04:50.569 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.569+0000 7fa5c7769740 1 Processor -- start 2024-04-22T07:04:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863b2ec0 0x5595863fa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863f1e40 0x5595863f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863aa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5c7769740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x559586315b10 con 0x5595863ac250 2024-04-22T07:04:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5c7769740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55958629e190 con 0x5595863f1e40 2024-04-22T07:04:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5c7769740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5595863b1250 con 0x5595863b2ec0 2024-04-22T07:04:50.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5c7769740 10 client.0 _ll_register_callbacks cb 0x5595862998b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863aa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46818/0 (socket says 172.21.15.44:46818) 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bed9d640 1 -- 172.21.15.44:0/313394761 learned_addr learned my addr 172.21.15.44:0/313394761 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5be59c640 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863b2ec0 0x5595863fa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bdd9b640 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863f1e40 0x5595863f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bed9d640 1 -- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863b2ec0 msgr2=0x5595863fa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863b2ec0 0x5595863fa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bed9d640 1 -- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863f1e40 msgr2=0x5595863f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863f1e40 0x5595863f3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bed9d640 1 -- 172.21.15.44:0/313394761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595860f26b0 con 0x5595863ac250 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bdd9b640 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863f1e40 0x5595863f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863aa170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000f8f0 tx=0x7fa5b000fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919b815caa281b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa59d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:50.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa59d7fa640 1 -- 172.21.15.44:0/313394761 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000dd70 con 0x5595863ac250 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.570+0000 7fa59d7fa640 1 -- 172.21.15.44:0/313394761 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa5b001aab0 con 0x5595863ac250 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa5c7769740 1 -- 172.21.15.44:0/313394761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b8084680 con 0x5595863ac250 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa59d7fa640 1 -- 172.21.15.44:0/313394761 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa5b001acb0 con 0x5595863ac250 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa5c7769740 10 client.4731 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa5c7769740 1 -- 172.21.15.44:0/313394761 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5595863f4340 con 0x5595863ac250 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa59d7fa640 1 -- 172.21.15.44:0/313394761 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b8084680 con 0x5595863ac250 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa59d7fa640 1 -- 172.21.15.44:0/313394761 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fa5b0021050 con 0x5595863ac250 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa5c7769740 10 client.4731 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa5c7769740 10 client.4731 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa5c7769740 20 client.4731 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa5c7769740 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55958640f5e0 0x5595864119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa5c7769740 4 client.4731 mds_command: new command op to 4324 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa5c7769740 1 -- 172.21.15.44:0/313394761 --> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559586323530 con 0x55958640f5e0 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.571+0000 7fa5be59c640 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55958640f5e0 0x5595864119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.572+0000 7fa59d7fa640 10 client.4731 ms_handle_connect on v2:172.21.15.70:6836/2872131879 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.572+0000 7fa5be59c640 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55958640f5e0 0x5595864119d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bda63e16e3df3f7 server_cookie=34dcb824ba08e338 in_seq=0 out_seq=0 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.572+0000 7fa59d7fa640 1 -- 172.21.15.44:0/313394761 <== mds.0 v2:172.21.15.70:6836/2872131879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559586323530 con 0x55958640f5e0 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.572+0000 7fa59d7fa640 10 client.4731 handle_command_reply: tid=0 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.572+0000 7fa5c7769740 1 client.4731 shutdown 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.572+0000 7fa5c7769740 1 -- 172.21.15.44:0/313394761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 msgr2=0x5595863aa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.572+0000 7fa5c7769740 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863aa170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000f8f0 tx=0x7fa5b000fdb0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.573+0000 7fa5c7769740 1 -- 172.21.15.44:0/313394761 shutdown_connections 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.573+0000 7fa5c7769740 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55958640f5e0 0x5595864119d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.573+0000 7fa5c7769740 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863aa170 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.573+0000 7fa5c7769740 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863f1e40 0x5595863f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.573+0000 7fa5c7769740 1 --2- 172.21.15.44:0/313394761 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863b2ec0 0x5595863fa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.573+0000 7fa59d7fa640 0 client.4731 ms_handle_reset on v2:172.21.15.70:6836/2872131879 2024-04-22T07:04:50.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.573+0000 7fa5c7769740 1 -- 172.21.15.44:0/313394761 >> 172.21.15.44:0/313394761 conn(0x5595863aeca0 msgr2=0x5595863f2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.573+0000 7fa5c7769740 1 -- 172.21.15.44:0/313394761 shutdown_connections 2024-04-22T07:04:50.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.573+0000 7fa5c7769740 1 -- 172.21.15.44:0/313394761 wait complete. 2024-04-22T07:04:50.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.573+0000 7fa5c7769740 20 client.4731 trim_cache size 0 max 16384 2024-04-22T07:04:50.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.591+0000 7fa5c7769740 1 Processor -- start 2024-04-22T07:04:50.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.591+0000 7fa5c7769740 1 -- start start 2024-04-22T07:04:50.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863ac250 0x5595863d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863b2ec0 0x5595863b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5c7769740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c01fd0b0 con 0x5595863f1e40 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5c7769740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c0150080 con 0x5595863ac250 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5c7769740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5595863f5f90 con 0x5595863b2ec0 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5bed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5bed9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46826/0 (socket says 172.21.15.44:46826) 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5bed9d640 1 -- 172.21.15.44:0/1152707477 learned_addr learned my addr 172.21.15.44:0/1152707477 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5bdd9b640 1 --2- 172.21.15.44:0/1152707477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863b2ec0 0x5595863b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5be59c640 1 --2- 172.21.15.44:0/1152707477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863ac250 0x5595863d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5bed9d640 1 -- 172.21.15.44:0/1152707477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863b2ec0 msgr2=0x5595863b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/1152707477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863b2ec0 0x5595863b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5bed9d640 1 -- 172.21.15.44:0/1152707477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863ac250 msgr2=0x5595863d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/1152707477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863ac250 0x5595863d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5bed9d640 1 -- 172.21.15.44:0/1152707477 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559586346e90 con 0x5595863f1e40 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.592+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/1152707477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863b19d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000f480 tx=0x7fa5b0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5caa31a7e397e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa59f7fe640 1 -- 172.21.15.44:0/1152707477 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000db70 con 0x5595863f1e40 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa59f7fe640 1 -- 172.21.15.44:0/1152707477 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b001ae50 con 0x5595863f1e40 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa59f7fe640 1 -- 172.21.15.44:0/1152707477 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000f270 con 0x5595863f1e40 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa5c7769740 1 -- 172.21.15.44:0/1152707477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 msgr2=0x5595863b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa5c7769740 1 --2- 172.21.15.44:0/1152707477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863b19d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000f480 tx=0x7fa5b0012000 comp rx=0 tx=0).stop 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa5c7769740 1 -- 172.21.15.44:0/1152707477 shutdown_connections 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa5c7769740 1 --2- 172.21.15.44:0/1152707477 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863b2ec0 0x5595863b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa5c7769740 1 --2- 172.21.15.44:0/1152707477 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863f1e40 0x5595863b19d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa5c7769740 1 --2- 172.21.15.44:0/1152707477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863ac250 0x5595863d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa5c7769740 1 -- 172.21.15.44:0/1152707477 >> 172.21.15.44:0/1152707477 conn(0x5595863aeca0 msgr2=0x5595863fa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.593 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa5c7769740 1 -- 172.21.15.44:0/1152707477 shutdown_connections 2024-04-22T07:04:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.593+0000 7fa5c7769740 1 -- 172.21.15.44:0/1152707477 wait complete. 2024-04-22T07:04:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.594+0000 7fa5c7769740 1 -- start start 2024-04-22T07:04:50.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.594+0000 7fa5c7769740 1 Processor -- start 2024-04-22T07:04:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863b2ec0 0x5595863d5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5c7769740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863f1e40 0x5595863d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5c7769740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5595863efe10 con 0x5595863ac250 2024-04-22T07:04:50.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5c7769740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x5595863ab600 con 0x5595863b2ec0 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5c7769740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5595863c8a30 con 0x5595863f1e40 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5c7769740 10 client.0 _ll_register_callbacks cb 0x7fa5b0008790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5be59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5be59c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863f8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46828/0 (socket says 172.21.15.44:46828) 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5be59c640 1 -- 172.21.15.44:0/2193819732 learned_addr learned my addr 172.21.15.44:0/2193819732 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5bdd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863b2ec0 0x5595863d5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863f1e40 0x5595863d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5be59c640 1 -- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863f1e40 msgr2=0x5595863d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5be59c640 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863f1e40 0x5595863d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5be59c640 1 -- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863b2ec0 msgr2=0x5595863d5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5be59c640 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863b2ec0 0x5595863d5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5be59c640 1 -- 172.21.15.44:0/2193819732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595863f8d40 con 0x5595863ac250 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5bed9d640 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863f1e40 0x5595863d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5be59c640 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863f8800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa5b807bc60 tx=0x7fa5b8044040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c22287e1ca4dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa59d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa59d7fa640 1 -- 172.21.15.44:0/2193819732 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8094040 con 0x5595863ac250 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa59d7fa640 1 -- 172.21.15.44:0/2193819732 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa5b80820c0 con 0x5595863ac250 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.595+0000 7fa5c7769740 1 -- 172.21.15.44:0/2193819732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b001ae50 con 0x5595863ac250 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa59d7fa640 1 -- 172.21.15.44:0/2193819732 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa5b8079bc0 con 0x5595863ac250 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa5c7769740 10 client.4737 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa5c7769740 1 -- 172.21.15.44:0/2193819732 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559586415090 con 0x5595863ac250 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa59d7fa640 1 -- 172.21.15.44:0/2193819732 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b001ae50 con 0x5595863ac250 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa59d7fa640 1 -- 172.21.15.44:0/2193819732 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fa5b8041070 con 0x5595863ac250 2024-04-22T07:04:50.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa5c7769740 10 client.4737 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:50.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa5c7769740 10 client.4737 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T07:04:50.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa5c7769740 20 client.4737 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:50.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa5c7769740 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x5595863d13b0 0x5595863d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa5c7769740 4 client.4737 mds_command: new command op to 4324 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:50.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa5c7769740 1 -- 172.21.15.44:0/2193819732 --> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] -- command(tid 0: {"prefix": "status"}) -- 0x55958631d3a0 con 0x5595863d13b0 2024-04-22T07:04:50.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa5bdd9b640 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x5595863d13b0 0x5595863d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa5bdd9b640 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x5595863d13b0 0x5595863d1790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2180b3cc0b3602a2 server_cookie=a5ed053f40357899 in_seq=0 out_seq=0 2024-04-22T07:04:50.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.596+0000 7fa59d7fa640 10 client.4737 ms_handle_connect on v2:172.21.15.70:6836/2872131879 2024-04-22T07:04:50.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa59d7fa640 1 -- 172.21.15.44:0/2193819732 <== mds.0 v2:172.21.15.70:6836/2872131879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55958631d3a0 con 0x5595863d13b0 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa59d7fa640 10 client.4737 handle_command_reply: tid=0 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 client.4737 shutdown 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 -- 172.21.15.44:0/2193819732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 msgr2=0x5595863f8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863f8800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa5b807bc60 tx=0x7fa5b8044040 comp rx=0 tx=0).stop 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 -- 172.21.15.44:0/2193819732 shutdown_connections 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5595863f1e40 0x5595863d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x5595863d13b0 0x5595863d1790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5595863b2ec0 0x5595863d5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 --2- 172.21.15.44:0/2193819732 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x5595863ac250 0x5595863f8800 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 -- 172.21.15.44:0/2193819732 >> 172.21.15.44:0/2193819732 conn(0x5595863aeca0 msgr2=0x5595863fa040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa59d7fa640 0 client.4737 ms_handle_reset on v2:172.21.15.70:6836/2872131879 2024-04-22T07:04:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 -- 172.21.15.44:0/2193819732 shutdown_connections 2024-04-22T07:04:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 1 -- 172.21.15.44:0/2193819732 wait complete. 2024-04-22T07:04:50.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c7769740 20 client.4737 trim_cache size 0 max 16384 2024-04-22T07:04:50.603 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:04:50.603 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "dc21e26c-6c65-477f-a367-ffb001b78b47", 2024-04-22T07:04:50.603 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-04-22T07:04:50.603 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4324, 2024-04-22T07:04:50.604 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T07:04:50.604 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T07:04:50.604 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T07:04:50.604 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 36.805282853999998, 2024-04-22T07:04:50.604 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:50.604 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 23, 2024-04-22T07:04:50.604 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 23, 2024-04-22T07:04:50.604 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 43.45156867 2024-04-22T07:04:50.604 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:04:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/614994968 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa5a006c920 msgr2=0x7fa5a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c4a9a640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa5a006c920 0x7fa5a006ede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa5c015f030 tx=0x7fa5b4009040 comp rx=0 tx=0).stop 2024-04-22T07:04:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/614994968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00cb3e0 msgr2=0x7fa5c015a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c4a9a640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00cb3e0 0x7fa5c015a390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8000c00 tx=0x7fa5b8004730 comp rx=0 tx=0).stop 2024-04-22T07:04:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/614994968 shutdown_connections 2024-04-22T07:04:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c4a9a640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fa5a006c920 0x7fa5a006ede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c4a9a640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fa5c0154f70 0x7fa5c0159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c4a9a640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c00cd6f0 0x7fa5c0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c4a9a640 1 --2- 172.21.15.44:0/614994968 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fa5c00cb3e0 0x7fa5c015a390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.597+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/614994968 >> 172.21.15.44:0/614994968 conn(0x7fa5c0067d40 msgr2=0x7fa5c0071f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.598+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/614994968 shutdown_connections 2024-04-22T07:04:50.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.598+0000 7fa5c4a9a640 1 -- 172.21.15.44:0/614994968 wait complete. 2024-04-22T07:04:50.608 INFO:teuthology.run_tasks:Running task sequential... 2024-04-22T07:04:50.617 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-22T07:04:50.618 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-22T07:04:50.618 INFO:teuthology.task.print:Enabling mgr modules 2024-04-22T07:04:50.618 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-22T07:04:50.641 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-22T07:04:50.641 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-22T07:04:50.642 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-22T07:04:50.642 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T07:04:50.642 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:50.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:50.642 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-22T07:04:50.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:50.658 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-04-22T07:04:50.725 INFO:teuthology.orchestra.run.smithi044.stderr:Cannot find device "ceph-brx" 2024-04-22T07:04:50.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:04:50.726 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-22T07:04:50.726 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-22T07:04:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.869+0000 7fb94e53b640 1 Processor -- start 2024-04-22T07:04:50.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.869+0000 7fb94e53b640 1 -- start start 2024-04-22T07:04:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb94e53b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb948154f70 0x7fb94815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb94e53b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9480cb3e0 0x7fb9480cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb94e53b640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb94815ee90 con 0x7fb9480cd6f0 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb94e53b640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fb9480cd260 con 0x7fb948154f70 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb94e53b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb94815f170 con 0x7fb9480cb3e0 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb947fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fb948154f70 0x7fb94815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb9477fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fb9480cd6f0 0x7fb9480c5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:33002/0 (socket says 172.21.15.44:33002) 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb9477fe640 1 -- 172.21.15.44:0/833088229 learned_addr learned my addr 172.21.15.44:0/833088229 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb94cad8640 1 --2- 172.21.15.44:0/833088229 >> v2:172.21.15.70:3300/0 conn(0x7fb9480cb3e0 0x7fb9480cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb947fff640 1 -- 172.21.15.44:0/833088229 >> v2:172.21.15.70:3300/0 conn(0x7fb9480cb3e0 msgr2=0x7fb9480cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb947fff640 1 --2- 172.21.15.44:0/833088229 >> v2:172.21.15.70:3300/0 conn(0x7fb9480cb3e0 0x7fb9480cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb947fff640 1 -- 172.21.15.44:0/833088229 >> v1:172.21.15.44:6789/0 conn(0x7fb9480cd6f0 legacy=0x7fb9480c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb947fff640 1 -- 172.21.15.44:0/833088229 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9480cc930 con 0x7fb948154f70 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.870+0000 7fb946ffd640 1 -- 172.21.15.44:0/833088229 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3974205018 0 0) 0x7fb94815ee90 con 0x7fb9480cd6f0 2024-04-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.871+0000 7fb947fff640 1 --2- 172.21.15.44:0/833088229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 0x7fb94815ad30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb9480633d0 tx=0x7fb93002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567104eb52314dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.871+0000 7fb946ffd640 1 -- 172.21.15.44:0/833088229 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930034070 con 0x7fb948154f70 2024-04-22T07:04:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.871+0000 7fb946ffd640 1 -- 172.21.15.44:0/833088229 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb930030610 con 0x7fb948154f70 2024-04-22T07:04:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.871+0000 7fb946ffd640 1 -- 172.21.15.44:0/833088229 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300309e0 con 0x7fb948154f70 2024-04-22T07:04:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.872+0000 7fb94e53b640 1 -- 172.21.15.44:0/833088229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 msgr2=0x7fb94815ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.872+0000 7fb94e53b640 1 --2- 172.21.15.44:0/833088229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 0x7fb94815ad30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb9480633d0 tx=0x7fb93002b960 comp rx=0 tx=0).stop 2024-04-22T07:04:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.872+0000 7fb94e53b640 1 -- 172.21.15.44:0/833088229 shutdown_connections 2024-04-22T07:04:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.872+0000 7fb94e53b640 1 --2- 172.21.15.44:0/833088229 >> v2:172.21.15.70:3300/0 conn(0x7fb9480cb3e0 0x7fb9480cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.872+0000 7fb94e53b640 1 --2- 172.21.15.44:0/833088229 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 0x7fb94815ad30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.872+0000 7fb94e53b640 1 -- 172.21.15.44:0/833088229 >> 172.21.15.44:0/833088229 conn(0x7fb948067d40 msgr2=0x7fb94806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.872+0000 7fb94e53b640 1 -- 172.21.15.44:0/833088229 shutdown_connections 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.872+0000 7fb94e53b640 1 -- 172.21.15.44:0/833088229 wait complete. 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.873+0000 7fb94e53b640 1 Processor -- start 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.873+0000 7fb94e53b640 1 -- start start 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94e53b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9480cb3e0 0x7fb948156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94e53b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9480cd6f0 0x7fb948150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94e53b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 0x7fb948156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94e53b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb948155350 con 0x7fb948154f70 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94e53b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb9480ccef0 con 0x7fb9480cd6f0 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94e53b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb948158200 con 0x7fb9480cb3e0 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94cad8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 0x7fb948156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94cad8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 0x7fb948156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46848/0 (socket says 172.21.15.44:46848) 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94cad8640 1 -- 172.21.15.44:0/240432525 learned_addr learned my addr 172.21.15.44:0/240432525 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:50.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb947fff640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9480cb3e0 0x7fb948156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb9477fe640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9480cd6f0 0x7fb948150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94cad8640 1 -- 172.21.15.44:0/240432525 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9480cb3e0 msgr2=0x7fb948156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94cad8640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9480cb3e0 0x7fb948156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94cad8640 1 -- 172.21.15.44:0/240432525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9480cd6f0 msgr2=0x7fb948150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94cad8640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9480cd6f0 0x7fb948150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94cad8640 1 -- 172.21.15.44:0/240432525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb948151430 con 0x7fb948154f70 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb947fff640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9480cb3e0 0x7fb948156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb9477fe640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9480cd6f0 0x7fb948150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb94cad8640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 0x7fb948156df0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb940007e40 tx=0x7fb94000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb944ff9640 1 -- 172.21.15.44:0/240432525 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940014620 con 0x7fb948154f70 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.874+0000 7fb944ff9640 1 -- 172.21.15.44:0/240432525 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb940019070 con 0x7fb948154f70 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.875+0000 7fb944ff9640 1 -- 172.21.15.44:0/240432525 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940014a20 con 0x7fb948154f70 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.875+0000 7fb94e53b640 1 -- 172.21.15.44:0/240432525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9480c79f0 con 0x7fb948154f70 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.875+0000 7fb94e53b640 1 -- 172.21.15.44:0/240432525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9480c7fb0 con 0x7fb948154f70 2024-04-22T07:04:50.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.875+0000 7fb94e53b640 1 -- 172.21.15.44:0/240432525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9480cc500 con 0x7fb948154f70 2024-04-22T07:04:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.876+0000 7fb944ff9640 1 -- 172.21.15.44:0/240432525 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb940014bc0 con 0x7fb948154f70 2024-04-22T07:04:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.876+0000 7fb944ff9640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb92806c8f0 0x7fb92806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:50.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.877+0000 7fb947fff640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb92806c8f0 0x7fb92806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:50.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.877+0000 7fb944ff9640 1 -- 172.21.15.44:0/240432525 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb94008c410 con 0x7fb948154f70 2024-04-22T07:04:50.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.877+0000 7fb947fff640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb92806c8f0 0x7fb92806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb930036000 tx=0x7fb930008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:50.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:50.881+0000 7fb944ff9640 1 -- 172.21.15.44:0/240432525 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb94005a840 con 0x7fb948154f70 2024-04-22T07:04:51.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.111+0000 7fb94e53b640 1 -- 172.21.15.44:0/240432525 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb9480cbf90 con 0x7fb948154f70 2024-04-22T07:04:51.109 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.112+0000 7fb944ff9640 1 -- 172.21.15.44:0/240432525 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fb94005a140 con 0x7fb948154f70 2024-04-22T07:04:51.110 INFO:teuthology.orchestra.run.smithi044.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-22T07:04:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 -- 172.21.15.44:0/240432525 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb92806c8f0 msgr2=0x7fb92806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb92806c8f0 0x7fb92806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb930036000 tx=0x7fb930008030 comp rx=0 tx=0).stop 2024-04-22T07:04:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 -- 172.21.15.44:0/240432525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 msgr2=0x7fb948156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 0x7fb948156df0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb940007e40 tx=0x7fb94000e4b0 comp rx=0 tx=0).stop 2024-04-22T07:04:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 -- 172.21.15.44:0/240432525 shutdown_connections 2024-04-22T07:04:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb92806c8f0 0x7fb92806edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:51.112 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb948154f70 0x7fb948156df0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:51.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9480cd6f0 0x7fb948150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:51.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 --2- 172.21.15.44:0/240432525 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb9480cb3e0 0x7fb948156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:51.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 -- 172.21.15.44:0/240432525 >> 172.21.15.44:0/240432525 conn(0x7fb948067d40 msgr2=0x7fb9480c5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:51.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 -- 172.21.15.44:0/240432525 shutdown_connections 2024-04-22T07:04:51.113 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51.114+0000 7fb94e53b640 1 -- 172.21.15.44:0/240432525 wait complete. 2024-04-22T07:04:51.123 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-22T07:04:51.123 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T07:04:51.123 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi044.front.sepia.ceph.com 2024-04-22T07:04:51.123 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-22T07:04:51.123 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:51.123 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-22T07:04:51.123 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T07:04:51.123 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T07:04:51.123 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-22T07:04:51.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:51.123 DEBUG:teuthology.orchestra.run.smithi044:> ip addr 2024-04-22T07:04:51.140 INFO:teuthology.orchestra.run.smithi044.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T07:04:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T07:04:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T07:04:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T07:04:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 ::1/128 scope host 2024-04-22T07:04:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T07:04:51.140 INFO:teuthology.orchestra.run.smithi044.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T07:04:51.140 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8c brd ff:ff:ff:ff:ff:ff 2024-04-22T07:04:51.141 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f0 2024-04-22T07:04:51.141 INFO:teuthology.orchestra.run.smithi044.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T07:04:51.141 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:6b:f7:8d brd ff:ff:ff:ff:ff:ff 2024-04-22T07:04:51.141 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp7s0f1 2024-04-22T07:04:51.142 INFO:teuthology.orchestra.run.smithi044.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T07:04:51.142 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:72 brd ff:ff:ff:ff:ff:ff 2024-04-22T07:04:51.142 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f0 2024-04-22T07:04:51.142 INFO:teuthology.orchestra.run.smithi044.stdout: inet 172.21.15.44/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-22T07:04:51.142 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft 42658sec preferred_lft 42658sec 2024-04-22T07:04:51.142 INFO:teuthology.orchestra.run.smithi044.stdout: inet6 fe80::ec4:7aff:febd:1572/64 scope link noprefixroute 2024-04-22T07:04:51.143 INFO:teuthology.orchestra.run.smithi044.stdout: valid_lft forever preferred_lft forever 2024-04-22T07:04:51.143 INFO:teuthology.orchestra.run.smithi044.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T07:04:51.143 INFO:teuthology.orchestra.run.smithi044.stdout: link/ether 0c:c4:7a:bd:15:73 brd ff:ff:ff:ff:ff:ff 2024-04-22T07:04:51.143 INFO:teuthology.orchestra.run.smithi044.stdout: altname enp1s0f1 2024-04-22T07:04:51.144 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T07:04:51.144 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:04:51.144 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T07:04:51.144 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add name ceph-brx type bridge 2024-04-22T07:04:51.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr flush dev ceph-brx 2024-04-22T07:04:51.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set ceph-brx up 2024-04-22T07:04:51.145 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T07:04:51.145 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T07:04:51.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:04:51.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:51.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:51.322 DEBUG:teuthology.orchestra.run.smithi044:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T07:04:51.346 INFO:teuthology.orchestra.run.smithi044.stdout:1 2024-04-22T07:04:51.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:51.348 DEBUG:teuthology.orchestra.run.smithi044:> route 2024-04-22T07:04:51.416 INFO:teuthology.orchestra.run.smithi044.stdout:Kernel IP routing table 2024-04-22T07:04:51.416 INFO:teuthology.orchestra.run.smithi044.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T07:04:51.416 INFO:teuthology.orchestra.run.smithi044.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-22T07:04:51.416 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-22T07:04:51.416 INFO:teuthology.orchestra.run.smithi044.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T07:04:51.417 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:04:51.417 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T07:04:51.417 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-22T07:04:51.417 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-22T07:04:51.417 DEBUG:teuthology.orchestra.run.smithi044:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-22T07:04:51.417 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T07:04:51.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:04:51.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:51.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:51.686 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-22T07:04:51.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:51.702 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list-id 2024-04-22T07:04:51.757 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:04:51.757 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T07:04:51.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:04:51.757 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-22T07:04:51.758 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T07:04:51.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:04:51.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:51.878 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-22T07:04:51.878 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:04:51.878 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T07:04:51.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-22T07:04:51.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T07:04:51.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-22T07:04:51.878 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-22T07:04:51.879 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-22T07:04:51.879 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T07:04:51.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:04:52.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:52.013 DEBUG:teuthology.orchestra.run.smithi044:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T07:04:52.013 DEBUG:teuthology.orchestra.run.smithi044:> set -e 2024-04-22T07:04:52.013 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set brx.0 up 2024-04-22T07:04:52.013 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T07:04:52.013 DEBUG:teuthology.orchestra.run.smithi044:> ') 2024-04-22T07:04:52.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T07:04:52.086 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:52.092 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-22T07:04:52.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T07:04:52.092 DEBUG:teuthology.orchestra.run.smithi044:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:52.110 INFO:teuthology.orchestra.run.smithi044.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-22T07:04:52.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T07:04:52.111 DEBUG:teuthology.orchestra.run.smithi044:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:52.167 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-04-22T07:04:52.233 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T07:04:52.289 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-04-22T07:04:52.290 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T07:04:52.291 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T07:04:52.292 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-04-22T07:04:52.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-04-22T07:04:52.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-0 2024-04-22T07:04:52.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-1 2024-04-22T07:04:52.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-2 2024-04-22T07:04:52.292 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-3 2024-04-22T07:04:52.292 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-04-22T07:04:52.292 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:04:52.292 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:04:52.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:52.293 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-04-22T07:04:52.344 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-22T07:04:52.345 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-22T07:04:52.387 DEBUG:teuthology.orchestra.run.smithi044:> sudo modprobe fuse 2024-04-22T07:04:52.411 DEBUG:teuthology.orchestra.run.smithi044:> cat /proc/self/mounts | awk '{print $2}' 2024-04-22T07:04:52.467 INFO:teuthology.orchestra.run.smithi044.stdout:/proc 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/sys 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/dev 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/security 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/shm 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/pts 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/run 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/cgroup 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/pstore 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/bpf 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/ 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/selinux 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/proc/sys/fs/binfmt_misc 2024-04-22T07:04:52.468 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/hugepages 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/dev/mqueue 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/debug 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/tracing 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysusers.service 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/kernel/config 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/sys/fs/fuse/connections 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-sysctl.service 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/nfs/rpc_pipefs 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/run/user/1000 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-0 2024-04-22T07:04:52.469 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-1 2024-04-22T07:04:52.470 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-2 2024-04-22T07:04:52.470 INFO:teuthology.orchestra.run.smithi044.stdout:/var/lib/ceph/osd/ceph-3 2024-04-22T07:04:52.470 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns 2024-04-22T07:04:52.470 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:04:52.470 INFO:teuthology.orchestra.run.smithi044.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:04:52.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:52.470 DEBUG:teuthology.orchestra.run.smithi044:> ls /sys/fs/fuse/connections 2024-04-22T07:04:52.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:2024-04-22T07:04:52.504+0000 7f7cf243a480 -1 init, newargv = 0x564dbcc6ca00 newargc=15 2024-04-22T07:04:52.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[42876]: starting ceph client 2024-04-22T07:04:52.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi044.stderr:ceph-fuse[42876]: starting fuse 2024-04-22T07:04:52.518 INFO:teuthology.orchestra.run.smithi044.stdout:43 2024-04-22T07:04:52.518 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-22T07:04:52.518 DEBUG:teuthology.orchestra.run.smithi044:> sudo stdin-killer -- python3 -c ' 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> import glob 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> import re 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> import os 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> import subprocess 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> def _find_admin_socket(client_name): 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> files = glob.glob(asok_path) 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> # Given a non-glob path, it better be there 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> if "*" not in asok_path: 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> assert(len(files) == 1) 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> return files[0] 2024-04-22T07:04:52.519 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> for f in files: 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> contents = proc_f.read() 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> if mountpoint in contents: 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> return f 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> print(_find_admin_socket("client.0")) 2024-04-22T07:04:52.520 DEBUG:teuthology.orchestra.run.smithi044:> ' 2024-04-22T07:04:52.637 INFO:teuthology.orchestra.run.smithi044.stdout:/var/run/ceph/ceph-client.0.42876.asok 2024-04-22T07:04:52.640 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T07:04:52.647 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42876.asok 2024-04-22T07:04:52.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:52.648 DEBUG:teuthology.orchestra.run.smithi044:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42876.asok status 2024-04-22T07:04:52.784 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:04:52.784 INFO:teuthology.orchestra.run.smithi044.stdout: "metadata": { 2024-04-22T07:04:52.784 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_sha1": "86f7587a5a09af35f0895e1a2d08527638fae697", 2024-04-22T07:04:52.784 INFO:teuthology.orchestra.run.smithi044.stdout: "ceph_version": "ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev)", 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "entity_id": "0", 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "hostname": "smithi044", 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "pid": "42876", 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "root": "/" 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_count": 0, 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "dentry_pinned_count": 0, 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4752, 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "inst": { 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "name": { 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "client", 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "num": 4752 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T07:04:52.785 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2353523133 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": { 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: "type": "v1", 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: "addr": "192.168.0.1:0", 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: "nonce": 2353523133 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: "inst_str": "client.4752 192.168.0.1:0/2353523133", 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: "addr_str": "192.168.0.1:0/2353523133", 2024-04-22T07:04:52.786 INFO:teuthology.orchestra.run.smithi044.stdout: "inode_count": 1, 2024-04-22T07:04:52.787 INFO:teuthology.orchestra.run.smithi044.stdout: "mds_epoch": 17, 2024-04-22T07:04:52.787 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch": 23, 2024-04-22T07:04:52.787 INFO:teuthology.orchestra.run.smithi044.stdout: "osd_epoch_barrier": 0, 2024-04-22T07:04:52.787 INFO:teuthology.orchestra.run.smithi044.stdout: "blocklisted": false, 2024-04-22T07:04:52.787 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs" 2024-04-22T07:04:52.787 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:04:52.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:52.795 DEBUG:teuthology.orchestra.run.smithi044:> stat --file-system '--printf=%T 2024-04-22T07:04:52.795 DEBUG:teuthology.orchestra.run.smithi044:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:52.810 INFO:teuthology.orchestra.run.smithi044.stdout:fuseblk 2024-04-22T07:04:52.810 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:52.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:04:52.811 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:52.877 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-22T07:04:52.891 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:53.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.020+0000 7fd3c7def640 1 Processor -- start 2024-04-22T07:04:53.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.020+0000 7fd3c7def640 1 -- start start 2024-04-22T07:04:53.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c7def640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 0x7fd3c015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c7def640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd3c00cb3e0 0x7fd3c00cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c7def640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3c015ee90 con 0x7fd3c0154f70 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c7def640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fd3c00cd260 con 0x7fd3c00cb3e0 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c7def640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c015f170 con 0x7fd3c00cd6f0 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c638c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd3c00cb3e0 0x7fd3c00cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c638c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fd3c00cb3e0 0x7fd3c00cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46858/0 (socket says 172.21.15.44:46858) 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c638c640 1 -- 172.21.15.44:0/3405766012 learned_addr learned my addr 172.21.15.44:0/3405766012 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c538a640 1 --2- 172.21.15.44:0/3405766012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 0x7fd3c015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c638c640 1 -- 172.21.15.44:0/3405766012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 msgr2=0x7fd3c015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c638c640 1 --2- 172.21.15.44:0/3405766012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 0x7fd3c015ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c638c640 1 -- 172.21.15.44:0/3405766012 >> v1:172.21.15.70:6789/0 conn(0x7fd3c0154f70 legacy=0x7fd3c00c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c638c640 1 -- 172.21.15.44:0/3405766012 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c00cc930 con 0x7fd3c00cb3e0 2024-04-22T07:04:53.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.021+0000 7fd3c538a640 1 --2- 172.21.15.44:0/3405766012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 0x7fd3c015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:53.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.022+0000 7fd3c638c640 1 --2- 172.21.15.44:0/3405766012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c00cb890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009e50 tx=0x7fd3b002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4357d7abeac265bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.022+0000 7fd3c4b89640 1 -- 172.21.15.44:0/3405766012 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b002cb90 con 0x7fd3c00cb3e0 2024-04-22T07:04:53.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.022+0000 7fd3c4b89640 1 -- 172.21.15.44:0/3405766012 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b002cd30 con 0x7fd3c00cb3e0 2024-04-22T07:04:53.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.022+0000 7fd3c4b89640 1 -- 172.21.15.44:0/3405766012 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0034610 con 0x7fd3c00cb3e0 2024-04-22T07:04:53.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.023+0000 7fd3c7def640 1 -- 172.21.15.44:0/3405766012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c00cb3e0 msgr2=0x7fd3c00cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.023+0000 7fd3c7def640 1 --2- 172.21.15.44:0/3405766012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c00cb890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009e50 tx=0x7fd3b002bff0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.023+0000 7fd3c7def640 1 -- 172.21.15.44:0/3405766012 shutdown_connections 2024-04-22T07:04:53.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.023+0000 7fd3c7def640 1 --2- 172.21.15.44:0/3405766012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c00cb890 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.023+0000 7fd3c7def640 1 --2- 172.21.15.44:0/3405766012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 0x7fd3c015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.023+0000 7fd3c7def640 1 -- 172.21.15.44:0/3405766012 >> 172.21.15.44:0/3405766012 conn(0x7fd3c0067d40 msgr2=0x7fd3c006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:53.114 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.023+0000 7fd3c7def640 1 -- 172.21.15.44:0/3405766012 shutdown_connections 2024-04-22T07:04:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.023+0000 7fd3c7def640 1 -- 172.21.15.44:0/3405766012 wait complete. 2024-04-22T07:04:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.024+0000 7fd3c7def640 1 Processor -- start 2024-04-22T07:04:53.115 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.024+0000 7fd3c7def640 1 -- start start 2024-04-22T07:04:53.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c7def640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c01534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c7def640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 0x7fd3c0157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c7def640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c0154f70 0x7fd3c01f0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c7def640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c0155350 con 0x7fd3c0154f70 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c7def640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c00ccef0 con 0x7fd3c00cb3e0 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c7def640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fd3c014bc90 con 0x7fd3c00cd6f0 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c638c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c0154f70 0x7fd3c01f0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c638c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c0154f70 0x7fd3c01f0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46866/0 (socket says 172.21.15.44:46866) 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c638c640 1 -- 172.21.15.44:0/3331471784 learned_addr learned my addr 172.21.15.44:0/3331471784 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c538a640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 0x7fd3c0157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c5b8b640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c01534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c638c640 1 -- 172.21.15.44:0/3331471784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 msgr2=0x7fd3c0157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c638c640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 0x7fd3c0157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c638c640 1 -- 172.21.15.44:0/3331471784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3c00cb3e0 msgr2=0x7fd3c01534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.118 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c638c640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c01534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.025+0000 7fd3c638c640 1 -- 172.21.15.44:0/3331471784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c01f09d0 con 0x7fd3c0154f70 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.026+0000 7fd3c638c640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c0154f70 0x7fd3c01f0ed0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009f70 tx=0x7fd3b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.026+0000 7fd3be7fc640 1 -- 172.21.15.44:0/3331471784 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0043020 con 0x7fd3c0154f70 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.026+0000 7fd3be7fc640 1 -- 172.21.15.44:0/3331471784 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b003e450 con 0x7fd3c0154f70 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.026+0000 7fd3c7def640 1 -- 172.21.15.44:0/3331471784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c014bee0 con 0x7fd3c0154f70 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.026+0000 7fd3be7fc640 1 -- 172.21.15.44:0/3331471784 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0009510 con 0x7fd3c0154f70 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.026+0000 7fd3c7def640 1 -- 172.21.15.44:0/3331471784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c014c4a0 con 0x7fd3c0154f70 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.027+0000 7fd3c7def640 1 -- 172.21.15.44:0/3331471784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c01f0ba0 con 0x7fd3c0154f70 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.027+0000 7fd3be7fc640 1 -- 172.21.15.44:0/3331471784 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3b0046020 con 0x7fd3c0154f70 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.028+0000 7fd3be7fc640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd3ac06c920 0x7fd3ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.028+0000 7fd3c5b8b640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd3ac06c920 0x7fd3ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.028+0000 7fd3be7fc640 1 -- 172.21.15.44:0/3331471784 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd3b00adf00 con 0x7fd3c0154f70 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.028+0000 7fd3c5b8b640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd3ac06c920 0x7fd3ac06ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80016e0 tx=0x7fd3b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.032+0000 7fd3be7fc640 1 -- 172.21.15.44:0/3331471784 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3b004d070 con 0x7fd3c0154f70 2024-04-22T07:04:53.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.223+0000 7fd3c7def640 1 -- 172.21.15.44:0/3331471784 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3c00cc500 con 0x7fd3c0154f70 2024-04-22T07:04:53.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.224+0000 7fd3be7fc640 1 -- 172.21.15.44:0/3331471784 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16557 (secure 0 0 0) 0x7fd3b007bce0 con 0x7fd3c0154f70 2024-04-22T07:04:53.222 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:53.222 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":23,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:14.778262+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:53.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 -- 172.21.15.44:0/3331471784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd3ac06c920 msgr2=0x7fd3ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd3ac06c920 0x7fd3ac06ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80016e0 tx=0x7fd3b8009040 comp rx=0 tx=0).stop 2024-04-22T07:04:53.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 -- 172.21.15.44:0/3331471784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c0154f70 msgr2=0x7fd3c01f0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c0154f70 0x7fd3c01f0ed0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009f70 tx=0x7fd3b0004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:53.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 -- 172.21.15.44:0/3331471784 shutdown_connections 2024-04-22T07:04:53.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fd3ac06c920 0x7fd3ac06ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fd3c00cd6f0 0x7fd3c0157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fd3c0154f70 0x7fd3c01f0ed0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 --2- 172.21.15.44:0/3331471784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c01534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 -- 172.21.15.44:0/3331471784 >> 172.21.15.44:0/3331471784 conn(0x7fd3c0067d40 msgr2=0x7fd3c00cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:53.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 -- 172.21.15.44:0/3331471784 shutdown_connections 2024-04-22T07:04:53.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.226+0000 7fd3c7def640 1 -- 172.21.15.44:0/3331471784 wait complete. 2024-04-22T07:04:53.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:04:01.723984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:11.652423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:12.098854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:53.235 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:53.338 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.341+0000 7efe5fc5c640 1 Processor -- start 2024-04-22T07:04:53.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.341+0000 7efe5fc5c640 1 -- start start 2024-04-22T07:04:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5fc5c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efe5806d9b0 0x7efe58151530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5fc5c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efe58071770 0x7efe58151a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5fc5c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe580c5560 con 0x7efe5806d5d0 2024-04-22T07:04:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5fc5c640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7efe580c5840 con 0x7efe58071770 2024-04-22T07:04:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5fc5c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7efe580c5b20 con 0x7efe5806d9b0 2024-04-22T07:04:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5e1f9640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efe58071770 0x7efe58151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5e1f9640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7efe58071770 0x7efe58151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46876/0 (socket says 172.21.15.44:46876) 2024-04-22T07:04:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5e1f9640 1 -- 172.21.15.44:0/1630010585 learned_addr learned my addr 172.21.15.44:0/1630010585 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:53.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5d1f7640 1 --2- 172.21.15.44:0/1630010585 >> v2:172.21.15.70:3300/0 conn(0x7efe5806d9b0 0x7efe58151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5e1f9640 1 -- 172.21.15.44:0/1630010585 >> v2:172.21.15.70:3300/0 conn(0x7efe5806d9b0 msgr2=0x7efe58151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5e1f9640 1 --2- 172.21.15.44:0/1630010585 >> v2:172.21.15.70:3300/0 conn(0x7efe5806d9b0 0x7efe58151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5e1f9640 1 -- 172.21.15.44:0/1630010585 >> v1:172.21.15.70:6789/0 conn(0x7efe5806d5d0 legacy=0x7efe580c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.342+0000 7efe5e1f9640 1 -- 172.21.15.44:0/1630010585 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe58073130 con 0x7efe58071770 2024-04-22T07:04:53.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.343+0000 7efe5e1f9640 1 --2- 172.21.15.44:0/1630010585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe58071770 0x7efe58151a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe50009e50 tx=0x7efe5002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899d8e98e785dbee server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.343+0000 7efe5c9f6640 1 -- 172.21.15.44:0/1630010585 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5002cbb0 con 0x7efe58071770 2024-04-22T07:04:53.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.343+0000 7efe5c9f6640 1 -- 172.21.15.44:0/1630010585 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5002cd50 con 0x7efe58071770 2024-04-22T07:04:53.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.343+0000 7efe5c9f6640 1 -- 172.21.15.44:0/1630010585 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50034610 con 0x7efe58071770 2024-04-22T07:04:53.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.344+0000 7efe5fc5c640 1 -- 172.21.15.44:0/1630010585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe58071770 msgr2=0x7efe58151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.344+0000 7efe5fc5c640 1 --2- 172.21.15.44:0/1630010585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe58071770 0x7efe58151a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe50009e50 tx=0x7efe5002c010 comp rx=0 tx=0).stop 2024-04-22T07:04:53.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.344+0000 7efe5fc5c640 1 -- 172.21.15.44:0/1630010585 shutdown_connections 2024-04-22T07:04:53.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.344+0000 7efe5fc5c640 1 --2- 172.21.15.44:0/1630010585 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe58071770 0x7efe58151a70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.344+0000 7efe5fc5c640 1 --2- 172.21.15.44:0/1630010585 >> v2:172.21.15.70:3300/0 conn(0x7efe5806d9b0 0x7efe58151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.344+0000 7efe5fc5c640 1 -- 172.21.15.44:0/1630010585 >> 172.21.15.44:0/1630010585 conn(0x7efe580c9020 msgr2=0x7efe580c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.344+0000 7efe5fc5c640 1 -- 172.21.15.44:0/1630010585 shutdown_connections 2024-04-22T07:04:53.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.344+0000 7efe5fc5c640 1 -- 172.21.15.44:0/1630010585 wait complete. 2024-04-22T07:04:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.345+0000 7efe5fc5c640 1 Processor -- start 2024-04-22T07:04:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.345+0000 7efe5fc5c640 1 -- start start 2024-04-22T07:04:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5fc5c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe5806d5d0 0x7efe58141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5fc5c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe5806d9b0 0x7efe581e1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5fc5c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe58071770 0x7efe581f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5fc5c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7efe580c5200 con 0x7efe5806d5d0 2024-04-22T07:04:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5fc5c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efe580736f0 con 0x7efe58071770 2024-04-22T07:04:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5fc5c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7efe58157940 con 0x7efe5806d9b0 2024-04-22T07:04:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d9f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe5806d5d0 0x7efe58141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.345 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d9f8640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe5806d5d0 0x7efe58141850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46886/0 (socket says 172.21.15.44:46886) 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d9f8640 1 -- 172.21.15.44:0/2037062300 learned_addr learned my addr 172.21.15.44:0/2037062300 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5e1f9640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe58071770 0x7efe581f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d1f7640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe5806d9b0 0x7efe581e1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d9f8640 1 -- 172.21.15.44:0/2037062300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe5806d9b0 msgr2=0x7efe581e1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d9f8640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe5806d9b0 0x7efe581e1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d9f8640 1 -- 172.21.15.44:0/2037062300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe58071770 msgr2=0x7efe581f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d9f8640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe58071770 0x7efe581f4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d9f8640 1 -- 172.21.15.44:0/2037062300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5814feb0 con 0x7efe5806d5d0 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d1f7640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe5806d9b0 0x7efe581e1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5d9f8640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe5806d5d0 0x7efe58141850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe5400c430 tx=0x7efe5400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe4e7fc640 1 -- 172.21.15.44:0/2037062300 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe540052d0 con 0x7efe5806d5d0 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe4e7fc640 1 -- 172.21.15.44:0/2037062300 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe54005490 con 0x7efe5806d5d0 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe4e7fc640 1 -- 172.21.15.44:0/2037062300 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54007260 con 0x7efe5806d5d0 2024-04-22T07:04:53.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.346+0000 7efe5fc5c640 1 -- 172.21.15.44:0/2037062300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe58150150 con 0x7efe5806d5d0 2024-04-22T07:04:53.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.347+0000 7efe5fc5c640 1 -- 172.21.15.44:0/2037062300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe58150710 con 0x7efe5806d5d0 2024-04-22T07:04:53.347 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.348+0000 7efe5fc5c640 1 -- 172.21.15.44:0/2037062300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe58068b00 con 0x7efe5806d5d0 2024-04-22T07:04:53.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.348+0000 7efe4e7fc640 1 -- 172.21.15.44:0/2037062300 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe54007400 con 0x7efe5806d5d0 2024-04-22T07:04:53.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.348+0000 7efe4e7fc640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7efe3c06c920 0x7efe3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.348+0000 7efe5d1f7640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7efe3c06c920 0x7efe3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.348+0000 7efe4e7fc640 1 -- 172.21.15.44:0/2037062300 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7efe5408b130 con 0x7efe5806d5d0 2024-04-22T07:04:53.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.349+0000 7efe5d1f7640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7efe3c06c920 0x7efe3c06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe580c4e00 tx=0x7efe48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.353+0000 7efe4e7fc640 1 -- 172.21.15.44:0/2037062300 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe54058de0 con 0x7efe5806d5d0 2024-04-22T07:04:53.537 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.539+0000 7efe5fc5c640 1 -- 172.21.15.44:0/2037062300 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe580c3ec0 con 0x7efe5806d5d0 2024-04-22T07:04:53.538 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.540+0000 7efe4e7fc640 1 -- 172.21.15.44:0/2037062300 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7efe5405d0e0 con 0x7efe5806d5d0 2024-04-22T07:04:53.538 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T07:04:53.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 -- 172.21.15.44:0/2037062300 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7efe3c06c920 msgr2=0x7efe3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7efe3c06c920 0x7efe3c06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe580c4e00 tx=0x7efe48008040 comp rx=0 tx=0).stop 2024-04-22T07:04:53.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 -- 172.21.15.44:0/2037062300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe5806d5d0 msgr2=0x7efe58141850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe5806d5d0 0x7efe58141850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe5400c430 tx=0x7efe5400c8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 -- 172.21.15.44:0/2037062300 shutdown_connections 2024-04-22T07:04:53.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7efe3c06c920 0x7efe3c06ede0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7efe5806d9b0 0x7efe581e1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe58071770 0x7efe581f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.540 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 --2- 172.21.15.44:0/2037062300 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7efe5806d5d0 0x7efe58141850 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 -- 172.21.15.44:0/2037062300 >> 172.21.15.44:0/2037062300 conn(0x7efe580c9020 msgr2=0x7efe58143ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:53.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 -- 172.21.15.44:0/2037062300 shutdown_connections 2024-04-22T07:04:53.541 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.542+0000 7efe5fc5c640 1 -- 172.21.15.44:0/2037062300 wait complete. 2024-04-22T07:04:53.551 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:04:53.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.656+0000 7fb88abce640 1 Processor -- start 2024-04-22T07:04:53.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.656+0000 7fb88abce640 1 -- start start 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.656+0000 7fb88abce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb8841552c0 0x7fb8840b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.656+0000 7fb88abce640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8840caec0 0x7fb8840d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.656+0000 7fb88abce640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8841556a0 con 0x7fb8840cc920 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.656+0000 7fb88abce640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fb88415b160 con 0x7fb8841552c0 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.656+0000 7fb88abce640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb88415b440 con 0x7fb8840caec0 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.657+0000 7fb88896a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb8841552c0 0x7fb8840b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.657+0000 7fb883fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8840caec0 0x7fb8840d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.657+0000 7fb88916b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb8840cc920 0x7fb8840cd310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:54908/0 (socket says 172.21.15.44:54908) 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.657+0000 7fb88916b640 1 -- 172.21.15.44:0/1783028622 learned_addr learned my addr 172.21.15.44:0/1783028622 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.657+0000 7fb88896a640 1 -- 172.21.15.44:0/1783028622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8840caec0 msgr2=0x7fb8840d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:53.655 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.657+0000 7fb88896a640 1 --2- 172.21.15.44:0/1783028622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8840caec0 0x7fb8840d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.657+0000 7fb88896a640 1 -- 172.21.15.44:0/1783028622 >> v1:172.21.15.70:6789/0 conn(0x7fb8840cc920 legacy=0x7fb8840cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.657+0000 7fb88896a640 1 -- 172.21.15.44:0/1783028622 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88415b720 con 0x7fb8841552c0 2024-04-22T07:04:53.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.658+0000 7fb88896a640 1 --2- 172.21.15.44:0/1783028622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8841552c0 0x7fb8840b32a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb8780097a0 tx=0x7fb87802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f14d439c49fba5c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.658+0000 7fb8837fe640 1 -- 172.21.15.44:0/1783028622 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878035020 con 0x7fb8841552c0 2024-04-22T07:04:53.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.658+0000 7fb8837fe640 1 -- 172.21.15.44:0/1783028622 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb878031810 con 0x7fb8841552c0 2024-04-22T07:04:53.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.658+0000 7fb8837fe640 1 -- 172.21.15.44:0/1783028622 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878031be0 con 0x7fb8841552c0 2024-04-22T07:04:53.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.659+0000 7fb88abce640 1 -- 172.21.15.44:0/1783028622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8841552c0 msgr2=0x7fb8840b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.659+0000 7fb88abce640 1 --2- 172.21.15.44:0/1783028622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8841552c0 0x7fb8840b32a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb8780097a0 tx=0x7fb87802b8b0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.659+0000 7fb88abce640 1 -- 172.21.15.44:0/1783028622 shutdown_connections 2024-04-22T07:04:53.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.659+0000 7fb88abce640 1 --2- 172.21.15.44:0/1783028622 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8840caec0 0x7fb8840d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.659+0000 7fb88abce640 1 --2- 172.21.15.44:0/1783028622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8841552c0 0x7fb8840b32a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.659+0000 7fb88abce640 1 -- 172.21.15.44:0/1783028622 >> 172.21.15.44:0/1783028622 conn(0x7fb8840676b0 msgr2=0x7fb884067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:53.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.659+0000 7fb88abce640 1 -- 172.21.15.44:0/1783028622 shutdown_connections 2024-04-22T07:04:53.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.659+0000 7fb88abce640 1 -- 172.21.15.44:0/1783028622 wait complete. 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.660+0000 7fb88abce640 1 Processor -- start 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.660+0000 7fb88abce640 1 -- start start 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88abce640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8840caec0 0x7fb88415e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88abce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840cc920 0x7fb8841e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88abce640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8841552c0 0x7fb8841e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88abce640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fb8840c6a00 con 0x7fb8840caec0 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88abce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb88415bce0 con 0x7fb8840cc920 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88abce640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fb8840cda20 con 0x7fb8841552c0 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88896a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8840caec0 0x7fb88415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88896a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8840caec0 0x7fb88415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46898/0 (socket says 172.21.15.44:46898) 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88896a640 1 -- 172.21.15.44:0/1832282203 learned_addr learned my addr 172.21.15.44:0/1832282203 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb883fff640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840cc920 0x7fb8841e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88916b640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8841552c0 0x7fb8841e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88896a640 1 -- 172.21.15.44:0/1832282203 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8841552c0 msgr2=0x7fb8841e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88896a640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8841552c0 0x7fb8841e4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88896a640 1 -- 172.21.15.44:0/1832282203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840cc920 msgr2=0x7fb8841e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88896a640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840cc920 0x7fb8841e49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88896a640 1 -- 172.21.15.44:0/1832282203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8841f1050 con 0x7fb8840caec0 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.661+0000 7fb88896a640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8840caec0 0x7fb88415e720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb878034040 tx=0x7fb8780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.662+0000 7fb8817fa640 1 -- 172.21.15.44:0/1832282203 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878040070 con 0x7fb8840caec0 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.662+0000 7fb8817fa640 1 -- 172.21.15.44:0/1832282203 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb878039cf0 con 0x7fb8840caec0 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.662+0000 7fb88abce640 1 -- 172.21.15.44:0/1832282203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8841f1330 con 0x7fb8840caec0 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.662+0000 7fb8817fa640 1 -- 172.21.15.44:0/1832282203 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878035040 con 0x7fb8840caec0 2024-04-22T07:04:53.661 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.662+0000 7fb88abce640 1 -- 172.21.15.44:0/1832282203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8841f18f0 con 0x7fb8840caec0 2024-04-22T07:04:53.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.663+0000 7fb88abce640 1 -- 172.21.15.44:0/1832282203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8841f1b30 con 0x7fb8840caec0 2024-04-22T07:04:53.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.663+0000 7fb8817fa640 1 -- 172.21.15.44:0/1832282203 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb878038020 con 0x7fb8840caec0 2024-04-22T07:04:53.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.664+0000 7fb8817fa640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb86806c8f0 0x7fb86806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.664+0000 7fb8817fa640 1 -- 172.21.15.44:0/1832282203 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb8780ad650 con 0x7fb8840caec0 2024-04-22T07:04:53.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.664+0000 7fb883fff640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb86806c8f0 0x7fb86806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.663 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.664+0000 7fb883fff640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb86806c8f0 0x7fb86806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb8840c8fa0 tx=0x7fb870009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.666 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.668+0000 7fb8817fa640 1 -- 172.21.15.44:0/1832282203 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb87807b3a0 con 0x7fb8840caec0 2024-04-22T07:04:53.853 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.856+0000 7fb88abce640 1 -- 172.21.15.44:0/1832282203 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb884072660 con 0x7fb8840caec0 2024-04-22T07:04:53.854 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.856+0000 7fb8817fa640 1 -- 172.21.15.44:0/1832282203 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fb884072660 con 0x7fb8840caec0 2024-04-22T07:04:53.854 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T07:04:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 -- 172.21.15.44:0/1832282203 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb86806c8f0 msgr2=0x7fb86806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb86806c8f0 0x7fb86806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb8840c8fa0 tx=0x7fb870009190 comp rx=0 tx=0).stop 2024-04-22T07:04:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 -- 172.21.15.44:0/1832282203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8840caec0 msgr2=0x7fb88415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.856 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8840caec0 0x7fb88415e720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb878034040 tx=0x7fb8780046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 -- 172.21.15.44:0/1832282203 shutdown_connections 2024-04-22T07:04:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fb86806c8f0 0x7fb86806edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fb8841552c0 0x7fb8841e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8840cc920 0x7fb8841e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 --2- 172.21.15.44:0/1832282203 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fb8840caec0 0x7fb88415e720 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 -- 172.21.15.44:0/1832282203 >> 172.21.15.44:0/1832282203 conn(0x7fb8840676b0 msgr2=0x7fb884067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 -- 172.21.15.44:0/1832282203 shutdown_connections 2024-04-22T07:04:53.857 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.858+0000 7fb88abce640 1 -- 172.21.15.44:0/1832282203 wait complete. 2024-04-22T07:04:53.867 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:04:53.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.973+0000 7f8116b15640 1 Processor -- start 2024-04-22T07:04:53.971 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.973+0000 7f8116b15640 1 -- start start 2024-04-22T07:04:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f8116b15640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8110152ca0 0x7f81100ca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f8116b15640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81100c8590 0x7f8110158980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f8116b15640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81100c51e0 0x7f81100c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f8116b15640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f8110158ec0 con 0x7f8110152ca0 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f8116b15640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f81100c9050 con 0x7f81100c8590 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f8116b15640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8110153080 con 0x7f81100c51e0 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f81148b1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8110152ca0 0x7f81100ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f81148b1640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8110152ca0 0x7f81100ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46910/0 (socket says 172.21.15.44:46910) 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f81148b1640 1 -- 172.21.15.44:0/1624911973 learned_addr learned my addr 172.21.15.44:0/1624911973 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f810ffff640 1 --2- 172.21.15.44:0/1624911973 >> v2:172.21.15.70:3300/0 conn(0x7f81100c8590 0x7f8110158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f81150b2640 1 --2- 172.21.15.44:0/1624911973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81100c51e0 0x7f81100c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f81148b1640 1 -- 172.21.15.44:0/1624911973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81100c51e0 msgr2=0x7f81100c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f81148b1640 1 --2- 172.21.15.44:0/1624911973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81100c51e0 0x7f81100c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f81148b1640 1 -- 172.21.15.44:0/1624911973 >> v2:172.21.15.70:3300/0 conn(0x7f81100c8590 msgr2=0x7f8110158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f81148b1640 1 --2- 172.21.15.44:0/1624911973 >> v2:172.21.15.70:3300/0 conn(0x7f81100c8590 0x7f8110158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.974+0000 7f81148b1640 1 -- 172.21.15.44:0/1624911973 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8110070050 con 0x7f8110152ca0 2024-04-22T07:04:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.975+0000 7f81148b1640 1 --2- 172.21.15.44:0/1624911973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110152ca0 0x7f81100ca000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f81100ca570 tx=0x7f810402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499b115220b149cd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.975+0000 7f810f7fe640 1 -- 172.21.15.44:0/1624911973 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810402e070 con 0x7f8110152ca0 2024-04-22T07:04:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.975+0000 7f810f7fe640 1 -- 172.21.15.44:0/1624911973 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8104037070 con 0x7f8110152ca0 2024-04-22T07:04:53.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.975+0000 7f810f7fe640 1 -- 172.21.15.44:0/1624911973 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104031a00 con 0x7f8110152ca0 2024-04-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.976+0000 7f8116b15640 1 -- 172.21.15.44:0/1624911973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110152ca0 msgr2=0x7f81100ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.976+0000 7f8116b15640 1 --2- 172.21.15.44:0/1624911973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110152ca0 0x7f81100ca000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f81100ca570 tx=0x7f810402b880 comp rx=0 tx=0).stop 2024-04-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.976+0000 7f8116b15640 1 -- 172.21.15.44:0/1624911973 shutdown_connections 2024-04-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.976+0000 7f8116b15640 1 --2- 172.21.15.44:0/1624911973 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81100c51e0 0x7f81100c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.976+0000 7f8116b15640 1 --2- 172.21.15.44:0/1624911973 >> v2:172.21.15.70:3300/0 conn(0x7f81100c8590 0x7f8110158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.976+0000 7f8116b15640 1 --2- 172.21.15.44:0/1624911973 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8110152ca0 0x7f81100ca000 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.976+0000 7f8116b15640 1 -- 172.21.15.44:0/1624911973 >> 172.21.15.44:0/1624911973 conn(0x7f81100718a0 msgr2=0x7f8110071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.976+0000 7f8116b15640 1 -- 172.21.15.44:0/1624911973 shutdown_connections 2024-04-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.976+0000 7f8116b15640 1 -- 172.21.15.44:0/1624911973 wait complete. 2024-04-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.977+0000 7f8116b15640 1 Processor -- start 2024-04-22T07:04:53.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.977+0000 7f8116b15640 1 -- start start 2024-04-22T07:04:53.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f8116b15640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81100c51e0 0x7f81101e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f8116b15640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81100c8590 0x7f8110157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f8116b15640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8110152ca0 0x7f8110157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f8116b15640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f81100c9ae0 con 0x7f81100c51e0 2024-04-22T07:04:53.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f8116b15640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f81100c9800 con 0x7f8110152ca0 2024-04-22T07:04:53.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f8116b15640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81100c8cf0 con 0x7f81100c8590 2024-04-22T07:04:53.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f81148b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81100c51e0 0x7f81101e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f81148b1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81100c51e0 0x7f81101e2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46912/0 (socket says 172.21.15.44:46912) 2024-04-22T07:04:53.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f81148b1640 1 -- 172.21.15.44:0/3356844007 learned_addr learned my addr 172.21.15.44:0/3356844007 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:53.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f81150b2640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8110152ca0 0x7f8110157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f810ffff640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81100c8590 0x7f8110157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f81148b1640 1 -- 172.21.15.44:0/3356844007 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81100c8590 msgr2=0x7f8110157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f81148b1640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81100c8590 0x7f8110157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f81148b1640 1 -- 172.21.15.44:0/3356844007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8110152ca0 msgr2=0x7f8110157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:53.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f81148b1640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8110152ca0 0x7f8110157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:53.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f81148b1640 1 -- 172.21.15.44:0/3356844007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81101e4af0 con 0x7f81100c51e0 2024-04-22T07:04:53.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.978+0000 7f81148b1640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81100c51e0 0x7f81101e2a10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f810402b8c0 tx=0x7f81040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.984+0000 7f810d7fa640 1 -- 172.21.15.44:0/3356844007 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104032670 con 0x7f81100c51e0 2024-04-22T07:04:53.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.984+0000 7f810d7fa640 1 -- 172.21.15.44:0/3356844007 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8104032830 con 0x7f81100c51e0 2024-04-22T07:04:53.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.984+0000 7f810d7fa640 1 -- 172.21.15.44:0/3356844007 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104032c20 con 0x7f81100c51e0 2024-04-22T07:04:53.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.984+0000 7f8116b15640 1 -- 172.21.15.44:0/3356844007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81101e4850 con 0x7f81100c51e0 2024-04-22T07:04:53.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.985+0000 7f8116b15640 1 -- 172.21.15.44:0/3356844007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8110158be0 con 0x7f81100c51e0 2024-04-22T07:04:53.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.986+0000 7f8116b15640 1 -- 172.21.15.44:0/3356844007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8110069210 con 0x7f81100c51e0 2024-04-22T07:04:53.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.986+0000 7f810d7fa640 1 -- 172.21.15.44:0/3356844007 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81040453f0 con 0x7f81100c51e0 2024-04-22T07:04:53.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.987+0000 7f810d7fa640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80f406c8f0 0x7f80f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:53.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.987+0000 7f810d7fa640 1 -- 172.21.15.44:0/3356844007 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f81040adb00 con 0x7f81100c51e0 2024-04-22T07:04:53.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.987+0000 7f810ffff640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80f406c8f0 0x7f80f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:53.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.987+0000 7f810ffff640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80f406c8f0 0x7f80f406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f811014efa0 tx=0x7f80fc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:53.990 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:53.992+0000 7f810d7fa640 1 -- 172.21.15.44:0/3356844007 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8104040070 con 0x7f81100c51e0 2024-04-22T07:04:54.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.180+0000 7f8116b15640 1 -- 172.21.15.44:0/3356844007 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f811015cb60 con 0x7f81100c51e0 2024-04-22T07:04:54.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.180+0000 7f810d7fa640 1 -- 172.21.15.44:0/3356844007 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7f810407b7d0 con 0x7f81100c51e0 2024-04-22T07:04:54.178 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T07:04:54.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 -- 172.21.15.44:0/3356844007 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80f406c8f0 msgr2=0x7f80f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80f406c8f0 0x7f80f406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f811014efa0 tx=0x7f80fc009190 comp rx=0 tx=0).stop 2024-04-22T07:04:54.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 -- 172.21.15.44:0/3356844007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81100c51e0 msgr2=0x7f81101e2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81100c51e0 0x7f81101e2a10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f810402b8c0 tx=0x7f81040046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 -- 172.21.15.44:0/3356844007 shutdown_connections 2024-04-22T07:04:54.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80f406c8f0 0x7f80f406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f81100c8590 0x7f8110157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8110152ca0 0x7f8110157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 --2- 172.21.15.44:0/3356844007 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f81100c51e0 0x7f81101e2a10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 -- 172.21.15.44:0/3356844007 >> 172.21.15.44:0/3356844007 conn(0x7f81100718a0 msgr2=0x7f81100cb610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:54.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 -- 172.21.15.44:0/3356844007 shutdown_connections 2024-04-22T07:04:54.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.182+0000 7f8116b15640 1 -- 172.21.15.44:0/3356844007 wait complete. 2024-04-22T07:04:54.190 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:54.290 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.293+0000 7fbf5268c640 1 Processor -- start 2024-04-22T07:04:54.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.293+0000 7fbf5268c640 1 -- start start 2024-04-22T07:04:54.292 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.293+0000 7fbf5268c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf4c154fe0 0x7fbf4c15e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.293+0000 7fbf5268c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c070d10 0x7fbf4c0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.293+0000 7fbf5268c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbf4c15b790 0x7fbf4c0676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.293+0000 7fbf5268c640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fbf4c15b300 con 0x7fbf4c15b790 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.293+0000 7fbf5268c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbf4c15f330 con 0x7fbf4c154fe0 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.293+0000 7fbf5268c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4c15ea90 con 0x7fbf4c070d10 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf50c29640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbf4c15b790 0x7fbf4c0676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf50c29640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fbf4c15b790 0x7fbf4c0676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46914/0 (socket says 172.21.15.44:46914) 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf50c29640 1 -- 172.21.15.44:0/3019805940 learned_addr learned my addr 172.21.15.44:0/3019805940 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf4b7fe640 1 --2- 172.21.15.44:0/3019805940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c070d10 0x7fbf4c0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf4bfff640 1 --2- 172.21.15.44:0/3019805940 >> v2:172.21.15.70:3300/0 conn(0x7fbf4c154fe0 0x7fbf4c15e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf50c29640 1 -- 172.21.15.44:0/3019805940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c070d10 msgr2=0x7fbf4c0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf50c29640 1 --2- 172.21.15.44:0/3019805940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c070d10 0x7fbf4c0c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf50c29640 1 -- 172.21.15.44:0/3019805940 >> v2:172.21.15.70:3300/0 conn(0x7fbf4c154fe0 msgr2=0x7fbf4c15e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:54.293 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf50c29640 1 --2- 172.21.15.44:0/3019805940 >> v2:172.21.15.70:3300/0 conn(0x7fbf4c154fe0 0x7fbf4c15e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf50c29640 1 -- 172.21.15.44:0/3019805940 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4c15ed70 con 0x7fbf4c15b790 2024-04-22T07:04:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf4b7fe640 1 --2- 172.21.15.44:0/3019805940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c070d10 0x7fbf4c0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.294+0000 7fbf4bfff640 1 --2- 172.21.15.44:0/3019805940 >> v2:172.21.15.70:3300/0 conn(0x7fbf4c154fe0 0x7fbf4c15e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf50c29640 1 --2- 172.21.15.44:0/3019805940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c15b790 0x7fbf4c0676b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c071360 tx=0x7fbf4402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6011c1195af34703 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf4affd640 1 -- 172.21.15.44:0/3019805940 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4402cb00 con 0x7fbf4c15b790 2024-04-22T07:04:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf4affd640 1 -- 172.21.15.44:0/3019805940 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4402cca0 con 0x7fbf4c15b790 2024-04-22T07:04:54.294 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf4affd640 1 -- 172.21.15.44:0/3019805940 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44034610 con 0x7fbf4c15b790 2024-04-22T07:04:54.296 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf5268c640 1 -- 172.21.15.44:0/3019805940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c15b790 msgr2=0x7fbf4c0676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf5268c640 1 --2- 172.21.15.44:0/3019805940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c15b790 0x7fbf4c0676b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c071360 tx=0x7fbf4402bf60 comp rx=0 tx=0).stop 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf5268c640 1 -- 172.21.15.44:0/3019805940 shutdown_connections 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf5268c640 1 --2- 172.21.15.44:0/3019805940 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c15b790 0x7fbf4c0676b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf5268c640 1 --2- 172.21.15.44:0/3019805940 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c070d10 0x7fbf4c0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf5268c640 1 --2- 172.21.15.44:0/3019805940 >> v2:172.21.15.70:3300/0 conn(0x7fbf4c154fe0 0x7fbf4c15e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.295+0000 7fbf5268c640 1 -- 172.21.15.44:0/3019805940 >> 172.21.15.44:0/3019805940 conn(0x7fbf4c0cd320 msgr2=0x7fbf4c0cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.296+0000 7fbf5268c640 1 -- 172.21.15.44:0/3019805940 shutdown_connections 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.296+0000 7fbf5268c640 1 -- 172.21.15.44:0/3019805940 wait complete. 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.297+0000 7fbf5268c640 1 Processor -- start 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.297+0000 7fbf5268c640 1 -- start start 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf5268c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c070d10 0x7fbf4c1e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf5268c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c154fe0 0x7fbf4c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf5268c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf4c15b790 0x7fbf4c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf5268c640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c15bc40 con 0x7fbf4c070d10 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf5268c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c15af90 con 0x7fbf4c15b790 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf5268c640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4c0c5480 con 0x7fbf4c154fe0 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf4bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c070d10 0x7fbf4c1e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf4bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c070d10 0x7fbf4c1e7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:46918/0 (socket says 172.21.15.44:46918) 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf4bfff640 1 -- 172.21.15.44:0/2036923963 learned_addr learned my addr 172.21.15.44:0/2036923963 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf50c29640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf4c15b790 0x7fbf4c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf4b7fe640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c154fe0 0x7fbf4c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf4bfff640 1 -- 172.21.15.44:0/2036923963 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c154fe0 msgr2=0x7fbf4c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf4bfff640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c154fe0 0x7fbf4c15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf4bfff640 1 -- 172.21.15.44:0/2036923963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf4c15b790 msgr2=0x7fbf4c15c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf4bfff640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf4c15b790 0x7fbf4c15c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf4bfff640 1 -- 172.21.15.44:0/2036923963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4c1f4800 con 0x7fbf4c070d10 2024-04-22T07:04:54.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf50c29640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf4c15b790 0x7fbf4c15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf4bfff640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c070d10 0x7fbf4c1e7c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00e810 tx=0x7fbf3c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf48ff9640 1 -- 172.21.15.44:0/2036923963 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00f040 con 0x7fbf4c070d10 2024-04-22T07:04:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf48ff9640 1 -- 172.21.15.44:0/2036923963 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf3c004030 con 0x7fbf4c070d10 2024-04-22T07:04:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf5268c640 1 -- 172.21.15.44:0/2036923963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4c1f4ae0 con 0x7fbf4c070d10 2024-04-22T07:04:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf48ff9640 1 -- 172.21.15.44:0/2036923963 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c004400 con 0x7fbf4c070d10 2024-04-22T07:04:54.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.298+0000 7fbf5268c640 1 -- 172.21.15.44:0/2036923963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4c0c59b0 con 0x7fbf4c070d10 2024-04-22T07:04:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.299+0000 7fbf5268c640 1 -- 172.21.15.44:0/2036923963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4c0c8050 con 0x7fbf4c070d10 2024-04-22T07:04:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.300+0000 7fbf48ff9640 1 -- 172.21.15.44:0/2036923963 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf3c00c6d0 con 0x7fbf4c070d10 2024-04-22T07:04:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.300+0000 7fbf48ff9640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbf3806c920 0x7fbf3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.300+0000 7fbf48ff9640 1 -- 172.21.15.44:0/2036923963 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7fbf3c08b9a0 con 0x7fbf4c070d10 2024-04-22T07:04:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.300+0000 7fbf4b7fe640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbf3806c920 0x7fbf3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.301+0000 7fbf4b7fe640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbf3806c920 0x7fbf3806ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c1536a0 tx=0x7fbf40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:54.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.305+0000 7fbf48ff9640 1 -- 172.21.15.44:0/2036923963 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf3c017020 con 0x7fbf4c070d10 2024-04-22T07:04:54.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.489+0000 7fbf5268c640 1 -- 172.21.15.44:0/2036923963 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf4c04fdc0 con 0x7fbf4c070d10 2024-04-22T07:04:54.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.491+0000 7fbf48ff9640 1 -- 172.21.15.44:0/2036923963 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16807 (secure 0 0 0) 0x7fbf3c059d40 con 0x7fbf4c070d10 2024-04-22T07:04:54.489 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:54.489 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":24,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:53.970931+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:54.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.492+0000 7fbf5268c640 1 -- 172.21.15.44:0/2036923963 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbf3806c920 msgr2=0x7fbf3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.492+0000 7fbf5268c640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbf3806c920 0x7fbf3806ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c1536a0 tx=0x7fbf40008040 comp rx=0 tx=0).stop 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.492+0000 7fbf5268c640 1 -- 172.21.15.44:0/2036923963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c070d10 msgr2=0x7fbf4c1e7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.492+0000 7fbf5268c640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c070d10 0x7fbf4c1e7c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00e810 tx=0x7fbf3c00ecd0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.493+0000 7fbf5268c640 1 -- 172.21.15.44:0/2036923963 shutdown_connections 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.493+0000 7fbf5268c640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fbf3806c920 0x7fbf3806ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.493+0000 7fbf5268c640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fbf4c154fe0 0x7fbf4c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.493+0000 7fbf5268c640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf4c15b790 0x7fbf4c15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.493+0000 7fbf5268c640 1 --2- 172.21.15.44:0/2036923963 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbf4c070d10 0x7fbf4c1e7c50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.493+0000 7fbf5268c640 1 -- 172.21.15.44:0/2036923963 >> 172.21.15.44:0/2036923963 conn(0x7fbf4c0cd320 msgr2=0x7fbf4c06caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.493+0000 7fbf5268c640 1 -- 172.21.15.44:0/2036923963 shutdown_connections 2024-04-22T07:04:54.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.493+0000 7fbf5268c640 1 -- 172.21.15.44:0/2036923963 wait complete. 2024-04-22T07:04:54.501 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:04:01.723984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:11.652423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:12.098854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:54.502 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.607+0000 7f9083614640 1 Processor -- start 2024-04-22T07:04:54.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.608+0000 7f9083614640 1 -- start start 2024-04-22T07:04:54.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.608+0000 7f9083614640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f907c06d9b0 0x7f907c151530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.608+0000 7f9083614640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f907c071770 0x7f907c151a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.608+0000 7f9083614640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f907c0c5560 con 0x7f907c06d5d0 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.608+0000 7f9083614640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f907c0c5840 con 0x7f907c06d9b0 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.608+0000 7f9083614640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f907c0c5b20 con 0x7f907c071770 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.609+0000 7f9080baf640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f907c06d9b0 0x7f907c151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.609+0000 7f9080baf640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f907c06d9b0 0x7f907c151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36686/0 (socket says 172.21.15.44:36686) 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.609+0000 7f9080baf640 1 -- 172.21.15.44:0/2399410270 learned_addr learned my addr 172.21.15.44:0/2399410270 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.609+0000 7f9081bb1640 1 --2- 172.21.15.44:0/2399410270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f907c071770 0x7f907c151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.609+0000 7f9080baf640 1 -- 172.21.15.44:0/2399410270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f907c071770 msgr2=0x7f907c151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.609+0000 7f9080baf640 1 --2- 172.21.15.44:0/2399410270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f907c071770 0x7f907c151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.609+0000 7f9080baf640 1 -- 172.21.15.44:0/2399410270 >> v1:172.21.15.70:6789/0 conn(0x7f907c06d5d0 legacy=0x7f907c0c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.609+0000 7f9080baf640 1 -- 172.21.15.44:0/2399410270 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c073130 con 0x7f907c06d9b0 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9080baf640 1 --2- 172.21.15.44:0/2399410270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c06d9b0 0x7f907c151530 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f906c0098a0 tx=0x7f906c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29808f4dc87ec2c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9073fff640 1 -- 172.21.15.44:0/2399410270 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c02f830 con 0x7f907c06d9b0 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9073fff640 1 -- 172.21.15.44:0/2399410270 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f906c02f9d0 con 0x7f907c06d9b0 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9073fff640 1 -- 172.21.15.44:0/2399410270 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c02fda0 con 0x7f907c06d9b0 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9083614640 1 -- 172.21.15.44:0/2399410270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c06d9b0 msgr2=0x7f907c151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9083614640 1 --2- 172.21.15.44:0/2399410270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c06d9b0 0x7f907c151530 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f906c0098a0 tx=0x7f906c02ba10 comp rx=0 tx=0).stop 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9083614640 1 -- 172.21.15.44:0/2399410270 shutdown_connections 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9083614640 1 --2- 172.21.15.44:0/2399410270 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f907c071770 0x7f907c151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9083614640 1 --2- 172.21.15.44:0/2399410270 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c06d9b0 0x7f907c151530 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9083614640 1 -- 172.21.15.44:0/2399410270 >> 172.21.15.44:0/2399410270 conn(0x7f907c0c9020 msgr2=0x7f907c0c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.610+0000 7f9083614640 1 -- 172.21.15.44:0/2399410270 shutdown_connections 2024-04-22T07:04:54.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.611+0000 7f9083614640 1 -- 172.21.15.44:0/2399410270 wait complete. 2024-04-22T07:04:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.612+0000 7f9083614640 1 Processor -- start 2024-04-22T07:04:54.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.612+0000 7f9083614640 1 -- start start 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.612+0000 7f9083614640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f907c06d5d0 0x7f907c0cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.612+0000 7f9083614640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c06d9b0 0x7f907c14ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.612+0000 7f9083614640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c071770 0x7f907c145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.612+0000 7f9083614640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0c5200 con 0x7f907c071770 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.612+0000 7f9083614640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0736f0 con 0x7f907c06d9b0 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.612+0000 7f9083614640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f907c157940 con 0x7f907c06d5d0 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.612+0000 7f9081bb1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c071770 0x7f907c145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.612+0000 7f9081bb1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c071770 0x7f907c145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36700/0 (socket says 172.21.15.44:36700) 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9081bb1640 1 -- 172.21.15.44:0/3977535750 learned_addr learned my addr 172.21.15.44:0/3977535750 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9080baf640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c06d9b0 0x7f907c14ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f90813b0640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f907c06d5d0 0x7f907c0cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9081bb1640 1 -- 172.21.15.44:0/3977535750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f907c06d5d0 msgr2=0x7f907c0cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9081bb1640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f907c06d5d0 0x7f907c0cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9081bb1640 1 -- 172.21.15.44:0/3977535750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c06d9b0 msgr2=0x7f907c14ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9081bb1640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c06d9b0 0x7f907c14ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9081bb1640 1 -- 172.21.15.44:0/3977535750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c156300 con 0x7f907c071770 2024-04-22T07:04:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9081bb1640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c071770 0x7f907c145b10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9078009fd0 tx=0x7f907800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9071ffb640 1 -- 172.21.15.44:0/3977535750 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078014420 con 0x7f907c071770 2024-04-22T07:04:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9071ffb640 1 -- 172.21.15.44:0/3977535750 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9078019070 con 0x7f907c071770 2024-04-22T07:04:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9083614640 1 -- 172.21.15.44:0/3977535750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c1565e0 con 0x7f907c071770 2024-04-22T07:04:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9071ffb640 1 -- 172.21.15.44:0/3977535750 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078014800 con 0x7f907c071770 2024-04-22T07:04:54.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.613+0000 7f9083614640 1 -- 172.21.15.44:0/3977535750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c156ba0 con 0x7f907c071770 2024-04-22T07:04:54.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.614+0000 7f9083614640 1 -- 172.21.15.44:0/3977535750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907c068b00 con 0x7f907c071770 2024-04-22T07:04:54.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.615+0000 7f9071ffb640 1 -- 172.21.15.44:0/3977535750 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90780149a0 con 0x7f907c071770 2024-04-22T07:04:54.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.615+0000 7f9071ffb640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f906006c920 0x7f906006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.615+0000 7f90813b0640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f906006c920 0x7f906006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.615+0000 7f9071ffb640 1 -- 172.21.15.44:0/3977535750 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f907808c420 con 0x7f907c071770 2024-04-22T07:04:54.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.616+0000 7f90813b0640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f906006c920 0x7f906006ede0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9074001640 tx=0x7f9074009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:54.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.619+0000 7f9071ffb640 1 -- 172.21.15.44:0/3977535750 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f907805a7d0 con 0x7f907c071770 2024-04-22T07:04:54.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.803+0000 7f9083614640 1 -- 172.21.15.44:0/3977535750 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f907c14b4d0 con 0x7f907c071770 2024-04-22T07:04:54.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.803+0000 7f9071ffb640 1 -- 172.21.15.44:0/3977535750 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f907805a0d0 con 0x7f907c071770 2024-04-22T07:04:54.801 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T07:04:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 -- 172.21.15.44:0/3977535750 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f906006c920 msgr2=0x7f906006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f906006c920 0x7f906006ede0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9074001640 tx=0x7f9074009040 comp rx=0 tx=0).stop 2024-04-22T07:04:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 -- 172.21.15.44:0/3977535750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c071770 msgr2=0x7f907c145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c071770 0x7f907c145b10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9078009fd0 tx=0x7f907800e4b0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 -- 172.21.15.44:0/3977535750 shutdown_connections 2024-04-22T07:04:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f906006c920 0x7f906006ede0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f907c071770 0x7f907c145b10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907c06d9b0 0x7f907c14ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 --2- 172.21.15.44:0/3977535750 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f907c06d5d0 0x7f907c0cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 -- 172.21.15.44:0/3977535750 >> 172.21.15.44:0/3977535750 conn(0x7f907c0c9020 msgr2=0x7f907c1581f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 -- 172.21.15.44:0/3977535750 shutdown_connections 2024-04-22T07:04:54.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.805+0000 7f9083614640 1 -- 172.21.15.44:0/3977535750 wait complete. 2024-04-22T07:04:54.814 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:04:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.919+0000 7f7c6b7ef640 1 Processor -- start 2024-04-22T07:04:54.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.920+0000 7f7c6b7ef640 1 -- start start 2024-04-22T07:04:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.920+0000 7f7c6b7ef640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7c640c3600 0x7f7c6406d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.920+0000 7f7c6b7ef640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c640c3b60 con 0x7f7c64152b90 2024-04-22T07:04:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.920+0000 7f7c6b7ef640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c640c41b0 con 0x7f7c6406d8a0 2024-04-22T07:04:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.920+0000 7f7c6b7ef640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f7c640c4490 con 0x7f7c640c3600 2024-04-22T07:04:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.921+0000 7f7c68d8a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7c640c3600 0x7f7c6406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.921+0000 7f7c6958b640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f7c64152b90 0x7f7c640c6b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60684/0 (socket says 172.21.15.44:60684) 2024-04-22T07:04:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.921+0000 7f7c6958b640 1 -- 172.21.15.44:0/718235885 learned_addr learned my addr 172.21.15.44:0/718235885 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.921+0000 7f7c68d8a640 1 -- 172.21.15.44:0/718235885 >> v1:172.21.15.70:6789/0 conn(0x7f7c6406d8a0 legacy=0x7f7c6406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.921+0000 7f7c68d8a640 1 -- 172.21.15.44:0/718235885 >> v1:172.21.15.44:6789/0 conn(0x7f7c64152b90 legacy=0x7f7c640c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.921+0000 7f7c68d8a640 1 -- 172.21.15.44:0/718235885 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c640c4770 con 0x7f7c640c3600 2024-04-22T07:04:54.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.922+0000 7f7c68d8a640 1 --2- 172.21.15.44:0/718235885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 0x7f7c6406d360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7c540097a0 tx=0x7f7c5402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07895875762a3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:54.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.922+0000 7f7c5bfff640 1 -- 172.21.15.44:0/718235885 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5402f810 con 0x7f7c640c3600 2024-04-22T07:04:54.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.922+0000 7f7c5bfff640 1 -- 172.21.15.44:0/718235885 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c5402f9b0 con 0x7f7c640c3600 2024-04-22T07:04:54.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.922+0000 7f7c5bfff640 1 -- 172.21.15.44:0/718235885 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5402fd80 con 0x7f7c640c3600 2024-04-22T07:04:54.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.922+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/718235885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 msgr2=0x7f7c6406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.922+0000 7f7c6b7ef640 1 --2- 172.21.15.44:0/718235885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 0x7f7c6406d360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7c540097a0 tx=0x7f7c5402b9f0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.922+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/718235885 shutdown_connections 2024-04-22T07:04:54.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.922+0000 7f7c6b7ef640 1 --2- 172.21.15.44:0/718235885 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 0x7f7c6406d360 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.922+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/718235885 >> 172.21.15.44:0/718235885 conn(0x7f7c640c9020 msgr2=0x7f7c640c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:54.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.923+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/718235885 shutdown_connections 2024-04-22T07:04:54.921 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.923+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/718235885 wait complete. 2024-04-22T07:04:54.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.924+0000 7f7c6b7ef640 1 Processor -- start 2024-04-22T07:04:54.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.924+0000 7f7c6b7ef640 1 -- start start 2024-04-22T07:04:54.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.924+0000 7f7c6b7ef640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c6406d8a0 0x7f7c64141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.924+0000 7f7c6b7ef640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 0x7f7c64146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.923 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.924+0000 7f7c6b7ef640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7c64152b90 0x7f7c640cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.924+0000 7f7c6b7ef640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7c64152f70 con 0x7f7c640c3600 2024-04-22T07:04:54.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.924+0000 7f7c6b7ef640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7c640c3e40 con 0x7f7c6406d8a0 2024-04-22T07:04:54.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.924+0000 7f7c6b7ef640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7c641530f0 con 0x7f7c64152b90 2024-04-22T07:04:54.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.924+0000 7f7c68d8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 0x7f7c64146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c68d8a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 0x7f7c64146ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36726/0 (socket says 172.21.15.44:36726) 2024-04-22T07:04:54.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c68d8a640 1 -- 172.21.15.44:0/2116992161 learned_addr learned my addr 172.21.15.44:0/2116992161 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:54.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c6958b640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c6406d8a0 0x7f7c64141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c69d8c640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7c64152b90 0x7f7c640cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.924 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c68d8a640 1 -- 172.21.15.44:0/2116992161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7c64152b90 msgr2=0x7f7c640cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c68d8a640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7c64152b90 0x7f7c640cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c68d8a640 1 -- 172.21.15.44:0/2116992161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c6406d8a0 msgr2=0x7f7c64141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:54.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c68d8a640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c6406d8a0 0x7f7c64141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:54.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c68d8a640 1 -- 172.21.15.44:0/2116992161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c64072cc0 con 0x7f7c640c3600 2024-04-22T07:04:54.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c68d8a640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 0x7f7c64146ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c5402c000 tx=0x7f7c54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:54.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c59ffb640 1 -- 172.21.15.44:0/2116992161 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54047070 con 0x7f7c640c3600 2024-04-22T07:04:54.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c59ffb640 1 -- 172.21.15.44:0/2116992161 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54037c60 con 0x7f7c640c3600 2024-04-22T07:04:54.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/2116992161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c64072fa0 con 0x7f7c640c3600 2024-04-22T07:04:54.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c59ffb640 1 -- 172.21.15.44:0/2116992161 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5402fe60 con 0x7f7c640c3600 2024-04-22T07:04:54.925 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.925+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/2116992161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c64073560 con 0x7f7c640c3600 2024-04-22T07:04:54.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.926+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/2116992161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c64068b00 con 0x7f7c640c3600 2024-04-22T07:04:54.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.927+0000 7f7c59ffb640 1 -- 172.21.15.44:0/2116992161 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c5404a070 con 0x7f7c640c3600 2024-04-22T07:04:54.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.927+0000 7f7c59ffb640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7c4806c8f0 0x7f7c4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:54.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.927+0000 7f7c59ffb640 1 -- 172.21.15.44:0/2116992161 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5825+0+0 (secure 0 0 0) 0x7f7c540ad9b0 con 0x7f7c640c3600 2024-04-22T07:04:54.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.927+0000 7f7c6958b640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7c4806c8f0 0x7f7c4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:54.926 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.928+0000 7f7c6958b640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7c4806c8f0 0x7f7c4806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c60005880 tx=0x7f7c60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:54.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:54.932+0000 7f7c59ffb640 1 -- 172.21.15.44:0/2116992161 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c54033030 con 0x7f7c640c3600 2024-04-22T07:04:55.116 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.118+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/2116992161 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7c64147ca0 con 0x7f7c640c3600 2024-04-22T07:04:55.117 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.119+0000 7f7c59ffb640 1 -- 172.21.15.44:0/2116992161 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f7c5407b680 con 0x7f7c640c3600 2024-04-22T07:04:55.117 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T07:04:55.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/2116992161 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7c4806c8f0 msgr2=0x7f7c4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7c4806c8f0 0x7f7c4806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c60005880 tx=0x7f7c60009040 comp rx=0 tx=0).stop 2024-04-22T07:04:55.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/2116992161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 msgr2=0x7f7c64146ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 0x7f7c64146ed0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c5402c000 tx=0x7f7c54004730 comp rx=0 tx=0).stop 2024-04-22T07:04:55.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/2116992161 shutdown_connections 2024-04-22T07:04:55.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7c4806c8f0 0x7f7c4806edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.119 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7c64152b90 0x7f7c640cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7c640c3600 0x7f7c64146ed0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 --2- 172.21.15.44:0/2116992161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c6406d8a0 0x7f7c64141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/2116992161 >> 172.21.15.44:0/2116992161 conn(0x7f7c640c9020 msgr2=0x7f7c640c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/2116992161 shutdown_connections 2024-04-22T07:04:55.120 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.121+0000 7f7c6b7ef640 1 -- 172.21.15.44:0/2116992161 wait complete. 2024-04-22T07:04:55.129 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:04:55.232 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.235+0000 7fad361bd640 1 Processor -- start 2024-04-22T07:04:55.233 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.235+0000 7fad361bd640 1 -- start start 2024-04-22T07:04:55.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad361bd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad30151de0 0x7fad300cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad361bd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fad3015c3c0 0x7fad3015c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad361bd640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3015cd70 con 0x7fad3015b950 2024-04-22T07:04:55.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad361bd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fad300c6770 con 0x7fad3015c3c0 2024-04-22T07:04:55.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad361bd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fad300c5880 con 0x7fad30151de0 2024-04-22T07:04:55.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad2ffff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fad3015c3c0 0x7fad3015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad2effd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fad3015b950 0x7fad300676b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60696/0 (socket says 172.21.15.44:60696) 2024-04-22T07:04:55.234 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad2effd640 1 -- 172.21.15.44:0/351127817 learned_addr learned my addr 172.21.15.44:0/351127817 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:55.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad2f7fe640 1 --2- 172.21.15.44:0/351127817 >> v2:172.21.15.70:3300/0 conn(0x7fad30151de0 0x7fad300cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad2ffff640 1 -- 172.21.15.44:0/351127817 >> v2:172.21.15.70:3300/0 conn(0x7fad30151de0 msgr2=0x7fad300cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:55.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad2ffff640 1 --2- 172.21.15.44:0/351127817 >> v2:172.21.15.70:3300/0 conn(0x7fad30151de0 0x7fad300cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad2ffff640 1 -- 172.21.15.44:0/351127817 >> v1:172.21.15.44:6789/0 conn(0x7fad3015b950 legacy=0x7fad300676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.235 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.236+0000 7fad2ffff640 1 -- 172.21.15.44:0/351127817 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad300c5b60 con 0x7fad3015c3c0 2024-04-22T07:04:55.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.237+0000 7fad2ffff640 1 --2- 172.21.15.44:0/351127817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 0x7fad3015c830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad300ce2c0 tx=0x7fad2802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4fc71d8bf5b175 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:55.236 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.237+0000 7fad2e7fc640 1 -- 172.21.15.44:0/351127817 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28039070 con 0x7fad3015c3c0 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.237+0000 7fad2e7fc640 1 -- 172.21.15.44:0/351127817 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2802cd00 con 0x7fad3015c3c0 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.237+0000 7fad2e7fc640 1 -- 172.21.15.44:0/351127817 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28034660 con 0x7fad3015c3c0 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.238+0000 7fad361bd640 1 -- 172.21.15.44:0/351127817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 msgr2=0x7fad3015c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.238+0000 7fad361bd640 1 --2- 172.21.15.44:0/351127817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 0x7fad3015c830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad300ce2c0 tx=0x7fad2802bf50 comp rx=0 tx=0).stop 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.238+0000 7fad361bd640 1 -- 172.21.15.44:0/351127817 shutdown_connections 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.238+0000 7fad361bd640 1 --2- 172.21.15.44:0/351127817 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 0x7fad3015c830 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.238+0000 7fad361bd640 1 --2- 172.21.15.44:0/351127817 >> v2:172.21.15.70:3300/0 conn(0x7fad30151de0 0x7fad300cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.238+0000 7fad361bd640 1 -- 172.21.15.44:0/351127817 >> 172.21.15.44:0/351127817 conn(0x7fad30073da0 msgr2=0x7fad300728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.238+0000 7fad361bd640 1 -- 172.21.15.44:0/351127817 shutdown_connections 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.238+0000 7fad361bd640 1 -- 172.21.15.44:0/351127817 wait complete. 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.239+0000 7fad361bd640 1 Processor -- start 2024-04-22T07:04:55.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.239+0000 7fad361bd640 1 -- start start 2024-04-22T07:04:55.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad361bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad30151de0 0x7fad301f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad361bd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad3015b950 0x7fad3015d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad361bd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 0x7fad3015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad361bd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fad300c6400 con 0x7fad3015c3c0 2024-04-22T07:04:55.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad361bd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fad300c6120 con 0x7fad30151de0 2024-04-22T07:04:55.240 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad361bd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fad3015bd90 con 0x7fad3015b950 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 0x7fad3015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 0x7fad3015ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36746/0 (socket says 172.21.15.44:36746) 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2ffff640 1 -- 172.21.15.44:0/3163360411 learned_addr learned my addr 172.21.15.44:0/3163360411 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2effd640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad3015b950 0x7fad3015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2f7fe640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad30151de0 0x7fad301f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2ffff640 1 -- 172.21.15.44:0/3163360411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad3015b950 msgr2=0x7fad3015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2ffff640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad3015b950 0x7fad3015d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2ffff640 1 -- 172.21.15.44:0/3163360411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad30151de0 msgr2=0x7fad301f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2ffff640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad30151de0 0x7fad301f17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2ffff640 1 -- 172.21.15.44:0/3163360411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad300b31a0 con 0x7fad3015c3c0 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2effd640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad3015b950 0x7fad3015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2f7fe640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad30151de0 0x7fad301f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad2ffff640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 0x7fad3015ecd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fad28009e50 tx=0x7fad28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:55.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad17fff640 1 -- 172.21.15.44:0/3163360411 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28039050 con 0x7fad3015c3c0 2024-04-22T07:04:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad17fff640 1 -- 172.21.15.44:0/3163360411 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad28045070 con 0x7fad3015c3c0 2024-04-22T07:04:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad17fff640 1 -- 172.21.15.44:0/3163360411 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2803e830 con 0x7fad3015c3c0 2024-04-22T07:04:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad361bd640 1 -- 172.21.15.44:0/3163360411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad30154830 con 0x7fad3015c3c0 2024-04-22T07:04:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.240+0000 7fad361bd640 1 -- 172.21.15.44:0/3163360411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad30154df0 con 0x7fad3015c3c0 2024-04-22T07:04:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.241+0000 7fad361bd640 1 -- 172.21.15.44:0/3163360411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad300694b0 con 0x7fad3015c3c0 2024-04-22T07:04:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.242+0000 7fad17fff640 1 -- 172.21.15.44:0/3163360411 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad2803e9d0 con 0x7fad3015c3c0 2024-04-22T07:04:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.242+0000 7fad17fff640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fad1c06c920 0x7fad1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.242+0000 7fad17fff640 1 -- 172.21.15.44:0/3163360411 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7fad280ae1c0 con 0x7fad3015c3c0 2024-04-22T07:04:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.242+0000 7fad2f7fe640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fad1c06c920 0x7fad1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.243+0000 7fad2f7fe640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fad1c06c920 0x7fad1c06ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fad20006fd0 tx=0x7fad20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:55.244 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.247+0000 7fad17fff640 1 -- 172.21.15.44:0/3163360411 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad300694b0 con 0x7fad3015c3c0 2024-04-22T07:04:55.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.432+0000 7fad361bd640 1 -- 172.21.15.44:0/3163360411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad30068b00 con 0x7fad3015c3c0 2024-04-22T07:04:55.430 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.432+0000 7fad17fff640 1 -- 172.21.15.44:0/3163360411 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fad2804e020 con 0x7fad3015c3c0 2024-04-22T07:04:55.431 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T07:04:55.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.434+0000 7fad361bd640 1 -- 172.21.15.44:0/3163360411 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fad1c06c920 msgr2=0x7fad1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.432 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.434+0000 7fad361bd640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fad1c06c920 0x7fad1c06ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fad20006fd0 tx=0x7fad20008040 comp rx=0 tx=0).stop 2024-04-22T07:04:55.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.435+0000 7fad361bd640 1 -- 172.21.15.44:0/3163360411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 msgr2=0x7fad3015ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.435+0000 7fad361bd640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 0x7fad3015ecd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fad28009e50 tx=0x7fad28004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:55.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.435+0000 7fad361bd640 1 -- 172.21.15.44:0/3163360411 shutdown_connections 2024-04-22T07:04:55.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.435+0000 7fad361bd640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fad1c06c920 0x7fad1c06ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.435+0000 7fad361bd640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fad3015b950 0x7fad3015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.435+0000 7fad361bd640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fad3015c3c0 0x7fad3015ecd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.435+0000 7fad361bd640 1 --2- 172.21.15.44:0/3163360411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad30151de0 0x7fad301f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.435+0000 7fad361bd640 1 -- 172.21.15.44:0/3163360411 >> 172.21.15.44:0/3163360411 conn(0x7fad30073da0 msgr2=0x7fad3015af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:55.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.435+0000 7fad361bd640 1 -- 172.21.15.44:0/3163360411 shutdown_connections 2024-04-22T07:04:55.433 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.435+0000 7fad361bd640 1 -- 172.21.15.44:0/3163360411 wait complete. 2024-04-22T07:04:55.443 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-22T07:04:55.443 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:55.543 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.546+0000 7fddbe306640 1 Processor -- start 2024-04-22T07:04:55.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.546+0000 7fddbe306640 1 -- start start 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddbe306640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fddb80caec0 0x7fddb80c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddbe306640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddb815e8b0 0x7fddb815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddbe306640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddb8155500 con 0x7fddb8155120 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddbe306640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fddb8072f90 con 0x7fddb80caec0 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddbe306640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fddb815f220 con 0x7fddb815e8b0 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddb77fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fddb80caec0 0x7fddb80c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddb77fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fddb80caec0 0x7fddb80c6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36750/0 (socket says 172.21.15.44:36750) 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddb77fe640 1 -- 172.21.15.44:0/3019907044 learned_addr learned my addr 172.21.15.44:0/3019907044 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddbc8a3640 1 --2- 172.21.15.44:0/3019907044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddb815e8b0 0x7fddb815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddb77fe640 1 -- 172.21.15.44:0/3019907044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddb815e8b0 msgr2=0x7fddb815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:55.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddb77fe640 1 --2- 172.21.15.44:0/3019907044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddb815e8b0 0x7fddb815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddb77fe640 1 -- 172.21.15.44:0/3019907044 >> v1:172.21.15.70:6789/0 conn(0x7fddb8155120 legacy=0x7fddb80d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.546 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.547+0000 7fddb77fe640 1 -- 172.21.15.44:0/3019907044 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb815b0d0 con 0x7fddb80caec0 2024-04-22T07:04:55.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.548+0000 7fddb77fe640 1 --2- 172.21.15.44:0/3019907044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 0x7fddb80c6380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fddac009870 tx=0x7fddac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e609f45998ba3a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:55.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.548+0000 7fddb6ffd640 1 -- 172.21.15.44:0/3019907044 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac034070 con 0x7fddb80caec0 2024-04-22T07:04:55.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.548+0000 7fddb6ffd640 1 -- 172.21.15.44:0/3019907044 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac030840 con 0x7fddb80caec0 2024-04-22T07:04:55.547 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.548+0000 7fddb6ffd640 1 -- 172.21.15.44:0/3019907044 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac030c10 con 0x7fddb80caec0 2024-04-22T07:04:55.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.548+0000 7fddbe306640 1 -- 172.21.15.44:0/3019907044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 msgr2=0x7fddb80c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.548+0000 7fddbe306640 1 --2- 172.21.15.44:0/3019907044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 0x7fddb80c6380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fddac009870 tx=0x7fddac02b980 comp rx=0 tx=0).stop 2024-04-22T07:04:55.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.549+0000 7fddbe306640 1 -- 172.21.15.44:0/3019907044 shutdown_connections 2024-04-22T07:04:55.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.549+0000 7fddbe306640 1 --2- 172.21.15.44:0/3019907044 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddb815e8b0 0x7fddb815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.549+0000 7fddbe306640 1 --2- 172.21.15.44:0/3019907044 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 0x7fddb80c6380 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.549+0000 7fddbe306640 1 -- 172.21.15.44:0/3019907044 >> 172.21.15.44:0/3019907044 conn(0x7fddb80676b0 msgr2=0x7fddb8067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:55.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.549+0000 7fddbe306640 1 -- 172.21.15.44:0/3019907044 shutdown_connections 2024-04-22T07:04:55.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.549+0000 7fddbe306640 1 -- 172.21.15.44:0/3019907044 wait complete. 2024-04-22T07:04:55.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.550+0000 7fddbe306640 1 Processor -- start 2024-04-22T07:04:55.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.550+0000 7fddbe306640 1 -- start start 2024-04-22T07:04:55.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddbe306640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 0x7fddb8159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddbe306640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddb8155120 0x7fddb814e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddbe306640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb815e8b0 0x7fddb81f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddbe306640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fddb815b970 con 0x7fddb80caec0 2024-04-22T07:04:55.550 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddbe306640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fddb815b690 con 0x7fddb815e8b0 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddbe306640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fddb815bc50 con 0x7fddb8155120 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 0x7fddb8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb7fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 0x7fddb8159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36758/0 (socket says 172.21.15.44:36758) 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb7fff640 1 -- 172.21.15.44:0/1532990702 learned_addr learned my addr 172.21.15.44:0/1532990702 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb77fe640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddb8155120 0x7fddb814e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddbc8a3640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb815e8b0 0x7fddb81f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb7fff640 1 -- 172.21.15.44:0/1532990702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddb8155120 msgr2=0x7fddb814e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb7fff640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddb8155120 0x7fddb814e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb7fff640 1 -- 172.21.15.44:0/1532990702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb815e8b0 msgr2=0x7fddb81f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb7fff640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb815e8b0 0x7fddb81f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb7fff640 1 -- 172.21.15.44:0/1532990702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb814f190 con 0x7fddb80caec0 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddbc8a3640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb815e8b0 0x7fddb81f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb7fff640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 0x7fddb8159620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdda800c430 tx=0x7fdda800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:55.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb4ff9640 1 -- 172.21.15.44:0/1532990702 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda80052d0 con 0x7fddb80caec0 2024-04-22T07:04:55.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb4ff9640 1 -- 172.21.15.44:0/1532990702 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda8005490 con 0x7fddb80caec0 2024-04-22T07:04:55.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddbe306640 1 -- 172.21.15.44:0/1532990702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb815de50 con 0x7fddb80caec0 2024-04-22T07:04:55.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddb4ff9640 1 -- 172.21.15.44:0/1532990702 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8007260 con 0x7fddb80caec0 2024-04-22T07:04:55.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.551+0000 7fddbe306640 1 -- 172.21.15.44:0/1532990702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb81f4a60 con 0x7fddb80caec0 2024-04-22T07:04:55.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.553+0000 7fddbe306640 1 -- 172.21.15.44:0/1532990702 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb815e370 con 0x7fddb80caec0 2024-04-22T07:04:55.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.553+0000 7fddb4ff9640 1 -- 172.21.15.44:0/1532990702 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdda8007400 con 0x7fddb80caec0 2024-04-22T07:04:55.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.553+0000 7fddb4ff9640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdda406c920 0x7fdda406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.553+0000 7fddb4ff9640 1 -- 172.21.15.44:0/1532990702 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7fdda808b130 con 0x7fddb80caec0 2024-04-22T07:04:55.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.553+0000 7fddb77fe640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdda406c920 0x7fdda406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.554+0000 7fddb77fe640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdda406c920 0x7fdda406ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fddac036000 tx=0x7fddac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:55.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.558+0000 7fddb4ff9640 1 -- 172.21.15.44:0/1532990702 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fddb815e370 con 0x7fddb80caec0 2024-04-22T07:04:55.740 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.742+0000 7fddbe306640 1 -- 172.21.15.44:0/1532990702 --> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fddb814f470 con 0x7fdda406c920 2024-04-22T07:04:55.746 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.748+0000 7fddb4ff9640 1 -- 172.21.15.44:0/1532990702 <== mgr.4101 v2:172.21.15.44:6832/1889438723 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251920 (secure 0 0 0) 0x7fddb814f470 con 0x7fdda406c920 2024-04-22T07:04:55.749 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:55.750 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 -- 172.21.15.44:0/1532990702 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdda406c920 msgr2=0x7fdda406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdda406c920 0x7fdda406ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fddac036000 tx=0x7fddac008030 comp rx=0 tx=0).stop 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 -- 172.21.15.44:0/1532990702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 msgr2=0x7fddb8159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 0x7fddb8159620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdda800c430 tx=0x7fdda800c8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 -- 172.21.15.44:0/1532990702 shutdown_connections 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdda406c920 0x7fdda406ede0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fddb8155120 0x7fddb814e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb815e8b0 0x7fddb81f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 --2- 172.21.15.44:0/1532990702 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fddb80caec0 0x7fddb8159620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 -- 172.21.15.44:0/1532990702 >> 172.21.15.44:0/1532990702 conn(0x7fddb80676b0 msgr2=0x7fddb80cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 -- 172.21.15.44:0/1532990702 shutdown_connections 2024-04-22T07:04:55.752 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.751+0000 7fddbe306640 1 -- 172.21.15.44:0/1532990702 wait complete. 2024-04-22T07:04:55.760 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-04-22T07:04:55.721228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598843,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":366,"num_write_kb":2480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240168,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734552,"statfs":{"total":773094113280,"available":772848181248,"internally_reserved":0,"allocated":3317760,"data_stored":1847125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":463,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001790"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961498+0000","last_change":"2024-04-22T07:04:13.800864+0000","last_active":"2024-04-22T07:04:14.961498+0000","last_peered":"2024-04-22T07:04:14.961498+0000","last_clean":"2024-04-22T07:04:14.961498+0000","last_became_active":"2024-04-22T07:04:13.800611+0000","last_became_peered":"2024-04-22T07:04:13.800611+0000","last_unstale":"2024-04-22T07:04:14.961498+0000","last_undegraded":"2024-04-22T07:04:14.961498+0000","last_fullsized":"2024-04-22T07:04:14.961498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:55.129127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056077+0000","last_change":"2024-04-22T07:04:13.798390+0000","last_active":"2024-04-22T07:04:15.056077+0000","last_peered":"2024-04-22T07:04:15.056077+0000","last_clean":"2024-04-22T07:04:15.056077+0000","last_became_active":"2024-04-22T07:04:13.797927+0000","last_became_peered":"2024-04-22T07:04:13.797927+0000","last_unstale":"2024-04-22T07:04:15.056077+0000","last_undegraded":"2024-04-22T07:04:15.056077+0000","last_fullsized":"2024-04-22T07:04:15.056077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056425+0000","last_change":"2024-04-22T07:04:13.795288+0000","last_active":"2024-04-22T07:04:15.056425+0000","last_peered":"2024-04-22T07:04:15.056425+0000","last_clean":"2024-04-22T07:04:15.056425+0000","last_became_active":"2024-04-22T07:04:13.794704+0000","last_became_peered":"2024-04-22T07:04:13.794704+0000","last_unstale":"2024-04-22T07:04:15.056425+0000","last_undegraded":"2024-04-22T07:04:15.056425+0000","last_fullsized":"2024-04-22T07:04:15.056425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783223+0000","last_change":"2024-04-22T07:04:13.801884+0000","last_active":"2024-04-22T07:04:14.783223+0000","last_peered":"2024-04-22T07:04:14.783223+0000","last_clean":"2024-04-22T07:04:14.783223+0000","last_became_active":"2024-04-22T07:04:13.801620+0000","last_became_peered":"2024-04-22T07:04:13.801620+0000","last_unstale":"2024-04-22T07:04:14.783223+0000","last_undegraded":"2024-04-22T07:04:14.783223+0000","last_fullsized":"2024-04-22T07:04:14.783223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961151+0000","last_change":"2024-04-22T07:04:13.798670+0000","last_active":"2024-04-22T07:04:14.961151+0000","last_peered":"2024-04-22T07:04:14.961151+0000","last_clean":"2024-04-22T07:04:14.961151+0000","last_became_active":"2024-04-22T07:04:13.798384+0000","last_became_peered":"2024-04-22T07:04:13.798384+0000","last_unstale":"2024-04-22T07:04:14.961151+0000","last_undegraded":"2024-04-22T07:04:14.961151+0000","last_fullsized":"2024-04-22T07:04:14.961151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783124+0000","last_change":"2024-04-22T07:04:13.796468+0000","last_active":"2024-04-22T07:04:14.783124+0000","last_peered":"2024-04-22T07:04:14.783124+0000","last_clean":"2024-04-22T07:04:14.783124+0000","last_became_active":"2024-04-22T07:04:13.796139+0000","last_became_peered":"2024-04-22T07:04:13.796139+0000","last_unstale":"2024-04-22T07:04:14.783124+0000","last_undegraded":"2024-04-22T07:04:14.783124+0000","last_fullsized":"2024-04-22T07:04:14.783124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961066+0000","last_change":"2024-04-22T07:04:13.801449+0000","last_active":"2024-04-22T07:04:14.961066+0000","last_peered":"2024-04-22T07:04:14.961066+0000","last_clean":"2024-04-22T07:04:14.961066+0000","last_became_active":"2024-04-22T07:04:13.799374+0000","last_became_peered":"2024-04-22T07:04:13.799374+0000","last_unstale":"2024-04-22T07:04:14.961066+0000","last_undegraded":"2024-04-22T07:04:14.961066+0000","last_fullsized":"2024-04-22T07:04:14.961066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054559+0000","last_change":"2024-04-22T07:04:13.802780+0000","last_active":"2024-04-22T07:04:15.054559+0000","last_peered":"2024-04-22T07:04:15.054559+0000","last_clean":"2024-04-22T07:04:15.054559+0000","last_became_active":"2024-04-22T07:04:13.802371+0000","last_became_peered":"2024-04-22T07:04:13.802371+0000","last_unstale":"2024-04-22T07:04:15.054559+0000","last_undegraded":"2024-04-22T07:04:15.054559+0000","last_fullsized":"2024-04-22T07:04:15.054559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961532+0000","last_change":"2024-04-22T07:04:13.792483+0000","last_active":"2024-04-22T07:04:14.961532+0000","last_peered":"2024-04-22T07:04:14.961532+0000","last_clean":"2024-04-22T07:04:14.961532+0000","last_became_active":"2024-04-22T07:04:13.792198+0000","last_became_peered":"2024-04-22T07:04:13.792198+0000","last_unstale":"2024-04-22T07:04:14.961532+0000","last_undegraded":"2024-04-22T07:04:14.961532+0000","last_fullsized":"2024-04-22T07:04:14.961532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:36.943428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782040+0000","last_change":"2024-04-22T07:04:13.797379+0000","last_active":"2024-04-22T07:04:14.782040+0000","last_peered":"2024-04-22T07:04:14.782040+0000","last_clean":"2024-04-22T07:04:14.782040+0000","last_became_active":"2024-04-22T07:04:13.797029+0000","last_became_peered":"2024-04-22T07:04:13.797029+0000","last_unstale":"2024-04-22T07:04:14.782040+0000","last_undegraded":"2024-04-22T07:04:14.782040+0000","last_fullsized":"2024-04-22T07:04:14.782040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781680+0000","last_change":"2024-04-22T07:04:13.789875+0000","last_active":"2024-04-22T07:04:14.781680+0000","last_peered":"2024-04-22T07:04:14.781680+0000","last_clean":"2024-04-22T07:04:14.781680+0000","last_became_active":"2024-04-22T07:04:13.789598+0000","last_became_peered":"2024-04-22T07:04:13.789598+0000","last_unstale":"2024-04-22T07:04:14.781680+0000","last_undegraded":"2024-04-22T07:04:14.781680+0000","last_fullsized":"2024-04-22T07:04:14.781680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055156+0000","last_change":"2024-04-22T07:04:13.791103+0000","last_active":"2024-04-22T07:04:15.055156+0000","last_peered":"2024-04-22T07:04:15.055156+0000","last_clean":"2024-04-22T07:04:15.055156+0000","last_became_active":"2024-04-22T07:04:13.790819+0000","last_became_peered":"2024-04-22T07:04:13.790819+0000","last_unstale":"2024-04-22T07:04:15.055156+0000","last_undegraded":"2024-04-22T07:04:15.055156+0000","last_fullsized":"2024-04-22T07:04:15.055156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:50.632020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055402+0000","last_change":"2024-04-22T07:04:13.802407+0000","last_active":"2024-04-22T07:04:15.055402+0000","last_peered":"2024-04-22T07:04:15.055402+0000","last_clean":"2024-04-22T07:04:15.055402+0000","last_became_active":"2024-04-22T07:04:13.801785+0000","last_became_peered":"2024-04-22T07:04:13.801785+0000","last_unstale":"2024-04-22T07:04:15.055402+0000","last_undegraded":"2024-04-22T07:04:15.055402+0000","last_fullsized":"2024-04-22T07:04:15.055402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055557+0000","last_change":"2024-04-22T07:04:13.795888+0000","last_active":"2024-04-22T07:04:15.055557+0000","last_peered":"2024-04-22T07:04:15.055557+0000","last_clean":"2024-04-22T07:04:15.055557+0000","last_became_active":"2024-04-22T07:04:13.795092+0000","last_became_peered":"2024-04-22T07:04:13.795092+0000","last_unstale":"2024-04-22T07:04:15.055557+0000","last_undegraded":"2024-04-22T07:04:15.055557+0000","last_fullsized":"2024-04-22T07:04:15.055557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:54.119208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055926+0000","last_change":"2024-04-22T07:04:13.797138+0000","last_active":"2024-04-22T07:04:15.055926+0000","last_peered":"2024-04-22T07:04:15.055926+0000","last_clean":"2024-04-22T07:04:15.055926+0000","last_became_active":"2024-04-22T07:04:13.796763+0000","last_became_peered":"2024-04-22T07:04:13.796763+0000","last_unstale":"2024-04-22T07:04:15.055926+0000","last_undegraded":"2024-04-22T07:04:15.055926+0000","last_fullsized":"2024-04-22T07:04:15.055926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782465+0000","last_change":"2024-04-22T07:04:13.803869+0000","last_active":"2024-04-22T07:04:14.782465+0000","last_peered":"2024-04-22T07:04:14.782465+0000","last_clean":"2024-04-22T07:04:14.782465+0000","last_became_active":"2024-04-22T07:04:13.803451+0000","last_became_peered":"2024-04-22T07:04:13.803451+0000","last_unstale":"2024-04-22T07:04:14.782465+0000","last_undegraded":"2024-04-22T07:04:14.782465+0000","last_fullsized":"2024-04-22T07:04:14.782465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783077+0000","last_change":"2024-04-22T07:04:13.795746+0000","last_active":"2024-04-22T07:04:14.783077+0000","last_peered":"2024-04-22T07:04:14.783077+0000","last_clean":"2024-04-22T07:04:14.783077+0000","last_became_active":"2024-04-22T07:04:13.794694+0000","last_became_peered":"2024-04-22T07:04:13.794694+0000","last_unstale":"2024-04-22T07:04:14.783077+0000","last_undegraded":"2024-04-22T07:04:14.783077+0000","last_fullsized":"2024-04-22T07:04:14.783077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054829+0000","last_change":"2024-04-22T07:04:13.790670+0000","last_active":"2024-04-22T07:04:15.054829+0000","last_peered":"2024-04-22T07:04:15.054829+0000","last_clean":"2024-04-22T07:04:15.054829+0000","last_became_active":"2024-04-22T07:04:13.790104+0000","last_became_peered":"2024-04-22T07:04:13.790104+0000","last_unstale":"2024-04-22T07:04:15.054829+0000","last_undegraded":"2024-04-22T07:04:15.054829+0000","last_fullsized":"2024-04-22T07:04:15.054829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:09.325971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961464+0000","last_change":"2024-04-22T07:04:13.799016+0000","last_active":"2024-04-22T07:04:14.961464+0000","last_peered":"2024-04-22T07:04:14.961464+0000","last_clean":"2024-04-22T07:04:14.961464+0000","last_became_active":"2024-04-22T07:04:13.798716+0000","last_became_peered":"2024-04-22T07:04:13.798716+0000","last_unstale":"2024-04-22T07:04:14.961464+0000","last_undegraded":"2024-04-22T07:04:14.961464+0000","last_fullsized":"2024-04-22T07:04:14.961464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:36.609382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781586+0000","last_change":"2024-04-22T07:04:13.797296+0000","last_active":"2024-04-22T07:04:14.781586+0000","last_peered":"2024-04-22T07:04:14.781586+0000","last_clean":"2024-04-22T07:04:14.781586+0000","last_became_active":"2024-04-22T07:04:13.796997+0000","last_became_peered":"2024-04-22T07:04:13.796997+0000","last_unstale":"2024-04-22T07:04:14.781586+0000","last_undegraded":"2024-04-22T07:04:14.781586+0000","last_fullsized":"2024-04-22T07:04:14.781586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055994+0000","last_change":"2024-04-22T07:04:13.799675+0000","last_active":"2024-04-22T07:04:15.055994+0000","last_peered":"2024-04-22T07:04:15.055994+0000","last_clean":"2024-04-22T07:04:15.055994+0000","last_became_active":"2024-04-22T07:04:13.799411+0000","last_became_peered":"2024-04-22T07:04:13.799411+0000","last_unstale":"2024-04-22T07:04:15.055994+0000","last_undegraded":"2024-04-22T07:04:15.055994+0000","last_fullsized":"2024-04-22T07:04:15.055994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783058+0000","last_change":"2024-04-22T07:04:13.796128+0000","last_active":"2024-04-22T07:04:14.783058+0000","last_peered":"2024-04-22T07:04:14.783058+0000","last_clean":"2024-04-22T07:04:14.783058+0000","last_became_active":"2024-04-22T07:04:13.795330+0000","last_became_peered":"2024-04-22T07:04:13.795330+0000","last_unstale":"2024-04-22T07:04:14.783058+0000","last_undegraded":"2024-04-22T07:04:14.783058+0000","last_fullsized":"2024-04-22T07:04:14.783058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055294+0000","last_change":"2024-04-22T07:04:13.800458+0000","last_active":"2024-04-22T07:04:15.055294+0000","last_peered":"2024-04-22T07:04:15.055294+0000","last_clean":"2024-04-22T07:04:15.055294+0000","last_became_active":"2024-04-22T07:04:13.800171+0000","last_became_peered":"2024-04-22T07:04:13.800171+0000","last_unstale":"2024-04-22T07:04:15.055294+0000","last_undegraded":"2024-04-22T07:04:15.055294+0000","last_fullsized":"2024-04-22T07:04:15.055294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782518+0000","last_change":"2024-04-22T07:04:13.802472+0000","last_active":"2024-04-22T07:04:14.782518+0000","last_peered":"2024-04-22T07:04:14.782518+0000","last_clean":"2024-04-22T07:04:14.782518+0000","last_became_active":"2024-04-22T07:04:13.802201+0000","last_became_peered":"2024-04-22T07:04:13.802201+0000","last_unstale":"2024-04-22T07:04:14.782518+0000","last_undegraded":"2024-04-22T07:04:14.782518+0000","last_fullsized":"2024-04-22T07:04:14.782518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055335+0000","last_change":"2024-04-22T07:04:13.795055+0000","last_active":"2024-04-22T07:04:15.055335+0000","last_peered":"2024-04-22T07:04:15.055335+0000","last_clean":"2024-04-22T07:04:15.055335+0000","last_became_active":"2024-04-22T07:04:13.794145+0000","last_became_peered":"2024-04-22T07:04:13.794145+0000","last_unstale":"2024-04-22T07:04:15.055335+0000","last_undegraded":"2024-04-22T07:04:15.055335+0000","last_fullsized":"2024-04-22T07:04:15.055335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782104+0000","last_change":"2024-04-22T07:04:13.799772+0000","last_active":"2024-04-22T07:04:14.782104+0000","last_peered":"2024-04-22T07:04:14.782104+0000","last_clean":"2024-04-22T07:04:14.782104+0000","last_became_active":"2024-04-22T07:04:13.799504+0000","last_became_peered":"2024-04-22T07:04:13.799504+0000","last_unstale":"2024-04-22T07:04:14.782104+0000","last_undegraded":"2024-04-22T07:04:14.782104+0000","last_fullsized":"2024-04-22T07:04:14.782104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056133+0000","last_change":"2024-04-22T07:04:13.804604+0000","last_active":"2024-04-22T07:04:15.056133+0000","last_peered":"2024-04-22T07:04:15.056133+0000","last_clean":"2024-04-22T07:04:15.056133+0000","last_became_active":"2024-04-22T07:04:13.803594+0000","last_became_peered":"2024-04-22T07:04:13.803594+0000","last_unstale":"2024-04-22T07:04:15.056133+0000","last_undegraded":"2024-04-22T07:04:15.056133+0000","last_fullsized":"2024-04-22T07:04:15.056133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055262+0000","last_change":"2024-04-22T07:04:13.795947+0000","last_active":"2024-04-22T07:04:15.055262+0000","last_peered":"2024-04-22T07:04:15.055262+0000","last_clean":"2024-04-22T07:04:15.055262+0000","last_became_active":"2024-04-22T07:04:13.795223+0000","last_became_peered":"2024-04-22T07:04:13.795223+0000","last_unstale":"2024-04-22T07:04:15.055262+0000","last_undegraded":"2024-04-22T07:04:15.055262+0000","last_fullsized":"2024-04-22T07:04:15.055262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056195+0000","last_change":"2024-04-22T07:04:13.804225+0000","last_active":"2024-04-22T07:04:15.056195+0000","last_peered":"2024-04-22T07:04:15.056195+0000","last_clean":"2024-04-22T07:04:15.056195+0000","last_became_active":"2024-04-22T07:04:13.803920+0000","last_became_peered":"2024-04-22T07:04:13.803920+0000","last_unstale":"2024-04-22T07:04:15.056195+0000","last_undegraded":"2024-04-22T07:04:15.056195+0000","last_fullsized":"2024-04-22T07:04:15.056195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783228+0000","last_change":"2024-04-22T07:04:13.796357+0000","last_active":"2024-04-22T07:04:14.783228+0000","last_peered":"2024-04-22T07:04:14.783228+0000","last_clean":"2024-04-22T07:04:14.783228+0000","last_became_active":"2024-04-22T07:04:13.795784+0000","last_became_peered":"2024-04-22T07:04:13.795784+0000","last_unstale":"2024-04-22T07:04:14.783228+0000","last_undegraded":"2024-04-22T07:04:14.783228+0000","last_fullsized":"2024-04-22T07:04:14.783228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054855+0000","last_change":"2024-04-22T07:04:13.784337+0000","last_active":"2024-04-22T07:04:15.054855+0000","last_peered":"2024-04-22T07:04:15.054855+0000","last_clean":"2024-04-22T07:04:15.054855+0000","last_became_active":"2024-04-22T07:04:13.783914+0000","last_became_peered":"2024-04-22T07:04:13.783914+0000","last_unstale":"2024-04-22T07:04:15.054855+0000","last_undegraded":"2024-04-22T07:04:15.054855+0000","last_fullsized":"2024-04-22T07:04:15.054855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:43:41.211395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781515+0000","last_change":"2024-04-22T07:04:13.806969+0000","last_active":"2024-04-22T07:04:14.781515+0000","last_peered":"2024-04-22T07:04:14.781515+0000","last_clean":"2024-04-22T07:04:14.781515+0000","last_became_active":"2024-04-22T07:04:13.806715+0000","last_became_peered":"2024-04-22T07:04:13.806715+0000","last_unstale":"2024-04-22T07:04:14.781515+0000","last_undegraded":"2024-04-22T07:04:14.781515+0000","last_fullsized":"2024-04-22T07:04:14.781515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054895+0000","last_change":"2024-04-22T07:04:13.794896+0000","last_active":"2024-04-22T07:04:15.054895+0000","last_peered":"2024-04-22T07:04:15.054895+0000","last_clean":"2024-04-22T07:04:15.054895+0000","last_became_active":"2024-04-22T07:04:13.794605+0000","last_became_peered":"2024-04-22T07:04:13.794605+0000","last_unstale":"2024-04-22T07:04:15.054895+0000","last_undegraded":"2024-04-22T07:04:15.054895+0000","last_fullsized":"2024-04-22T07:04:15.054895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:02.613130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.836956+0000","last_change":"2024-04-22T07:04:12.786738+0000","last_active":"2024-04-22T07:04:27.836956+0000","last_peered":"2024-04-22T07:04:27.836956+0000","last_clean":"2024-04-22T07:04:27.836956+0000","last_became_active":"2024-04-22T07:04:12.786413+0000","last_became_peered":"2024-04-22T07:04:12.786413+0000","last_unstale":"2024-04-22T07:04:27.836956+0000","last_undegraded":"2024-04-22T07:04:27.836956+0000","last_fullsized":"2024-04-22T07:04:27.836956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056196+0000","last_change":"2024-04-22T07:04:13.800525+0000","last_active":"2024-04-22T07:04:15.056196+0000","last_peered":"2024-04-22T07:04:15.056196+0000","last_clean":"2024-04-22T07:04:15.056196+0000","last_became_active":"2024-04-22T07:04:13.800254+0000","last_became_peered":"2024-04-22T07:04:13.800254+0000","last_unstale":"2024-04-22T07:04:15.056196+0000","last_undegraded":"2024-04-22T07:04:15.056196+0000","last_fullsized":"2024-04-22T07:04:15.056196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.873809+0000","last_change":"2024-04-22T07:04:12.786726+0000","last_active":"2024-04-22T07:04:39.873809+0000","last_peered":"2024-04-22T07:04:39.873809+0000","last_clean":"2024-04-22T07:04:39.873809+0000","last_became_active":"2024-04-22T07:04:12.786444+0000","last_became_peered":"2024-04-22T07:04:12.786444+0000","last_unstale":"2024-04-22T07:04:39.873809+0000","last_undegraded":"2024-04-22T07:04:39.873809+0000","last_fullsized":"2024-04-22T07:04:39.873809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:33.855518+0000","last_change":"2024-04-22T07:04:12.787357+0000","last_active":"2024-04-22T07:04:33.855518+0000","last_peered":"2024-04-22T07:04:33.855518+0000","last_clean":"2024-04-22T07:04:33.855518+0000","last_became_active":"2024-04-22T07:04:12.787066+0000","last_became_peered":"2024-04-22T07:04:12.787066+0000","last_unstale":"2024-04-22T07:04:33.855518+0000","last_undegraded":"2024-04-22T07:04:33.855518+0000","last_fullsized":"2024-04-22T07:04:33.855518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054713+0000","last_change":"2024-04-22T07:04:13.807697+0000","last_active":"2024-04-22T07:04:15.054713+0000","last_peered":"2024-04-22T07:04:15.054713+0000","last_clean":"2024-04-22T07:04:15.054713+0000","last_became_active":"2024-04-22T07:04:13.807327+0000","last_became_peered":"2024-04-22T07:04:13.807327+0000","last_unstale":"2024-04-22T07:04:15.054713+0000","last_undegraded":"2024-04-22T07:04:15.054713+0000","last_fullsized":"2024-04-22T07:04:15.054713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.873858+0000","last_change":"2024-04-22T07:04:12.787783+0000","last_active":"2024-04-22T07:04:39.873858+0000","last_peered":"2024-04-22T07:04:39.873858+0000","last_clean":"2024-04-22T07:04:39.873858+0000","last_became_active":"2024-04-22T07:04:12.786765+0000","last_became_peered":"2024-04-22T07:04:12.786765+0000","last_unstale":"2024-04-22T07:04:39.873858+0000","last_undegraded":"2024-04-22T07:04:39.873858+0000","last_fullsized":"2024-04-22T07:04:39.873858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055740+0000","last_change":"2024-04-22T07:04:13.804931+0000","last_active":"2024-04-22T07:04:15.055740+0000","last_peered":"2024-04-22T07:04:15.055740+0000","last_clean":"2024-04-22T07:04:15.055740+0000","last_became_active":"2024-04-22T07:04:13.804629+0000","last_became_peered":"2024-04-22T07:04:13.804629+0000","last_unstale":"2024-04-22T07:04:15.055740+0000","last_undegraded":"2024-04-22T07:04:15.055740+0000","last_fullsized":"2024-04-22T07:04:15.055740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.834749+0000","last_change":"2024-04-22T07:04:12.789279+0000","last_active":"2024-04-22T07:04:27.834749+0000","last_peered":"2024-04-22T07:04:27.834749+0000","last_clean":"2024-04-22T07:04:27.834749+0000","last_became_active":"2024-04-22T07:04:12.787705+0000","last_became_peered":"2024-04-22T07:04:12.787705+0000","last_unstale":"2024-04-22T07:04:27.834749+0000","last_undegraded":"2024-04-22T07:04:27.834749+0000","last_fullsized":"2024-04-22T07:04:27.834749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782812+0000","last_change":"2024-04-22T07:04:13.794292+0000","last_active":"2024-04-22T07:04:14.782812+0000","last_peered":"2024-04-22T07:04:14.782812+0000","last_clean":"2024-04-22T07:04:14.782812+0000","last_became_active":"2024-04-22T07:04:13.793986+0000","last_became_peered":"2024-04-22T07:04:13.793986+0000","last_unstale":"2024-04-22T07:04:14.782812+0000","last_undegraded":"2024-04-22T07:04:14.782812+0000","last_fullsized":"2024-04-22T07:04:14.782812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944030+0000","last_change":"2024-04-22T07:04:12.787064+0000","last_active":"2024-04-22T07:04:44.944030+0000","last_peered":"2024-04-22T07:04:44.944030+0000","last_clean":"2024-04-22T07:04:44.944030+0000","last_became_active":"2024-04-22T07:04:12.786430+0000","last_became_peered":"2024-04-22T07:04:12.786430+0000","last_unstale":"2024-04-22T07:04:44.944030+0000","last_undegraded":"2024-04-22T07:04:44.944030+0000","last_fullsized":"2024-04-22T07:04:44.944030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056732+0000","last_change":"2024-04-22T07:04:13.801337+0000","last_active":"2024-04-22T07:04:15.056732+0000","last_peered":"2024-04-22T07:04:15.056732+0000","last_clean":"2024-04-22T07:04:15.056732+0000","last_became_active":"2024-04-22T07:04:13.800916+0000","last_became_peered":"2024-04-22T07:04:13.800916+0000","last_unstale":"2024-04-22T07:04:15.056732+0000","last_undegraded":"2024-04-22T07:04:15.056732+0000","last_fullsized":"2024-04-22T07:04:15.056732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.834615+0000","last_change":"2024-04-22T07:04:12.787741+0000","last_active":"2024-04-22T07:04:27.834615+0000","last_peered":"2024-04-22T07:04:27.834615+0000","last_clean":"2024-04-22T07:04:27.834615+0000","last_became_active":"2024-04-22T07:04:12.787405+0000","last_became_peered":"2024-04-22T07:04:12.787405+0000","last_unstale":"2024-04-22T07:04:27.834615+0000","last_undegraded":"2024-04-22T07:04:27.834615+0000","last_fullsized":"2024-04-22T07:04:27.834615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055184+0000","last_change":"2024-04-22T07:04:13.795502+0000","last_active":"2024-04-22T07:04:15.055184+0000","last_peered":"2024-04-22T07:04:15.055184+0000","last_clean":"2024-04-22T07:04:15.055184+0000","last_became_active":"2024-04-22T07:04:13.795199+0000","last_became_peered":"2024-04-22T07:04:13.795199+0000","last_unstale":"2024-04-22T07:04:15.055184+0000","last_undegraded":"2024-04-22T07:04:15.055184+0000","last_fullsized":"2024-04-22T07:04:15.055184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:07.780008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'10","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.943628+0000","last_change":"2024-04-22T07:04:12.788572+0000","last_active":"2024-04-22T07:04:44.943628+0000","last_peered":"2024-04-22T07:04:44.943628+0000","last_clean":"2024-04-22T07:04:44.943628+0000","last_became_active":"2024-04-22T07:04:12.788286+0000","last_became_peered":"2024-04-22T07:04:12.788286+0000","last_unstale":"2024-04-22T07:04:44.943628+0000","last_undegraded":"2024-04-22T07:04:44.943628+0000","last_fullsized":"2024-04-22T07:04:44.943628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961160+0000","last_change":"2024-04-22T07:04:13.804435+0000","last_active":"2024-04-22T07:04:14.961160+0000","last_peered":"2024-04-22T07:04:14.961160+0000","last_clean":"2024-04-22T07:04:14.961160+0000","last_became_active":"2024-04-22T07:04:13.804170+0000","last_became_peered":"2024-04-22T07:04:13.804170+0000","last_unstale":"2024-04-22T07:04:14.961160+0000","last_undegraded":"2024-04-22T07:04:14.961160+0000","last_fullsized":"2024-04-22T07:04:14.961160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'9","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944342+0000","last_change":"2024-04-22T07:04:12.789905+0000","last_active":"2024-04-22T07:04:44.944342+0000","last_peered":"2024-04-22T07:04:44.944342+0000","last_clean":"2024-04-22T07:04:44.944342+0000","last_became_active":"2024-04-22T07:04:12.789503+0000","last_became_peered":"2024-04-22T07:04:12.789503+0000","last_unstale":"2024-04-22T07:04:44.944342+0000","last_undegraded":"2024-04-22T07:04:44.944342+0000","last_fullsized":"2024-04-22T07:04:44.944342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055741+0000","last_change":"2024-04-22T07:04:13.801490+0000","last_active":"2024-04-22T07:04:15.055741+0000","last_peered":"2024-04-22T07:04:15.055741+0000","last_clean":"2024-04-22T07:04:15.055741+0000","last_became_active":"2024-04-22T07:04:13.799356+0000","last_became_peered":"2024-04-22T07:04:13.799356+0000","last_unstale":"2024-04-22T07:04:15.055741+0000","last_undegraded":"2024-04-22T07:04:15.055741+0000","last_fullsized":"2024-04-22T07:04:15.055741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:32:04.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.877037+0000","last_change":"2024-04-22T07:04:12.781089+0000","last_active":"2024-04-22T07:04:39.877037+0000","last_peered":"2024-04-22T07:04:39.877037+0000","last_clean":"2024-04-22T07:04:39.877037+0000","last_became_active":"2024-04-22T07:04:12.780827+0000","last_became_peered":"2024-04-22T07:04:12.780827+0000","last_unstale":"2024-04-22T07:04:39.877037+0000","last_undegraded":"2024-04-22T07:04:39.877037+0000","last_fullsized":"2024-04-22T07:04:39.877037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961202+0000","last_change":"2024-04-22T07:04:13.801375+0000","last_active":"2024-04-22T07:04:14.961202+0000","last_peered":"2024-04-22T07:04:14.961202+0000","last_clean":"2024-04-22T07:04:14.961202+0000","last_became_active":"2024-04-22T07:04:13.799488+0000","last_became_peered":"2024-04-22T07:04:13.799488+0000","last_unstale":"2024-04-22T07:04:14.961202+0000","last_undegraded":"2024-04-22T07:04:14.961202+0000","last_fullsized":"2024-04-22T07:04:14.961202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.872946+0000","last_change":"2024-04-22T07:04:12.790758+0000","last_active":"2024-04-22T07:04:39.872946+0000","last_peered":"2024-04-22T07:04:39.872946+0000","last_clean":"2024-04-22T07:04:39.872946+0000","last_became_active":"2024-04-22T07:04:12.790444+0000","last_became_peered":"2024-04-22T07:04:12.790444+0000","last_unstale":"2024-04-22T07:04:39.872946+0000","last_undegraded":"2024-04-22T07:04:39.872946+0000","last_fullsized":"2024-04-22T07:04:39.872946+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782297+0000","last_change":"2024-04-22T07:04:13.805415+0000","last_active":"2024-04-22T07:04:14.782297+0000","last_peered":"2024-04-22T07:04:14.782297+0000","last_clean":"2024-04-22T07:04:14.782297+0000","last_became_active":"2024-04-22T07:04:13.805115+0000","last_became_peered":"2024-04-22T07:04:13.805115+0000","last_unstale":"2024-04-22T07:04:14.782297+0000","last_undegraded":"2024-04-22T07:04:14.782297+0000","last_fullsized":"2024-04-22T07:04:14.782297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'17","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.875047+0000","last_change":"2024-04-22T07:04:12.789877+0000","last_active":"2024-04-22T07:04:39.875047+0000","last_peered":"2024-04-22T07:04:39.875047+0000","last_clean":"2024-04-22T07:04:39.875047+0000","last_became_active":"2024-04-22T07:04:12.789546+0000","last_became_peered":"2024-04-22T07:04:12.789546+0000","last_unstale":"2024-04-22T07:04:39.875047+0000","last_undegraded":"2024-04-22T07:04:39.875047+0000","last_fullsized":"2024-04-22T07:04:39.875047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:04.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961082+0000","last_change":"2024-04-22T07:04:13.801803+0000","last_active":"2024-04-22T07:04:14.961082+0000","last_peered":"2024-04-22T07:04:14.961082+0000","last_clean":"2024-04-22T07:04:14.961082+0000","last_became_active":"2024-04-22T07:04:13.801362+0000","last_became_peered":"2024-04-22T07:04:13.801362+0000","last_unstale":"2024-04-22T07:04:14.961082+0000","last_undegraded":"2024-04-22T07:04:14.961082+0000","last_fullsized":"2024-04-22T07:04:14.961082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.802173+0000","last_change":"2024-04-22T07:04:12.787917+0000","last_active":"2024-04-22T07:04:18.802173+0000","last_peered":"2024-04-22T07:04:18.802173+0000","last_clean":"2024-04-22T07:04:18.802173+0000","last_became_active":"2024-04-22T07:04:12.787636+0000","last_became_peered":"2024-04-22T07:04:12.787636+0000","last_unstale":"2024-04-22T07:04:18.802173+0000","last_undegraded":"2024-04-22T07:04:18.802173+0000","last_fullsized":"2024-04-22T07:04:18.802173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961456+0000","last_change":"2024-04-22T07:04:13.802786+0000","last_active":"2024-04-22T07:04:14.961456+0000","last_peered":"2024-04-22T07:04:14.961456+0000","last_clean":"2024-04-22T07:04:14.961456+0000","last_became_active":"2024-04-22T07:04:13.801299+0000","last_became_peered":"2024-04-22T07:04:13.801299+0000","last_unstale":"2024-04-22T07:04:14.961456+0000","last_undegraded":"2024-04-22T07:04:14.961456+0000","last_fullsized":"2024-04-22T07:04:14.961456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:28.362520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944029+0000","last_change":"2024-04-22T07:04:12.788593+0000","last_active":"2024-04-22T07:04:44.944029+0000","last_peered":"2024-04-22T07:04:44.944029+0000","last_clean":"2024-04-22T07:04:44.944029+0000","last_became_active":"2024-04-22T07:04:12.788314+0000","last_became_peered":"2024-04-22T07:04:12.788314+0000","last_unstale":"2024-04-22T07:04:44.944029+0000","last_undegraded":"2024-04-22T07:04:44.944029+0000","last_fullsized":"2024-04-22T07:04:44.944029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:11:27.081266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961608+0000","last_change":"2024-04-22T07:04:13.800245+0000","last_active":"2024-04-22T07:04:14.961608+0000","last_peered":"2024-04-22T07:04:14.961608+0000","last_clean":"2024-04-22T07:04:14.961608+0000","last_became_active":"2024-04-22T07:04:13.799970+0000","last_became_peered":"2024-04-22T07:04:13.799970+0000","last_unstale":"2024-04-22T07:04:14.961608+0000","last_undegraded":"2024-04-22T07:04:14.961608+0000","last_fullsized":"2024-04-22T07:04:14.961608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:13.558356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.874909+0000","last_change":"2024-04-22T07:04:12.790003+0000","last_active":"2024-04-22T07:04:39.874909+0000","last_peered":"2024-04-22T07:04:39.874909+0000","last_clean":"2024-04-22T07:04:39.874909+0000","last_became_active":"2024-04-22T07:04:12.789708+0000","last_became_peered":"2024-04-22T07:04:12.789708+0000","last_unstale":"2024-04-22T07:04:39.874909+0000","last_undegraded":"2024-04-22T07:04:39.874909+0000","last_fullsized":"2024-04-22T07:04:39.874909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783345+0000","last_change":"2024-04-22T07:04:13.795082+0000","last_active":"2024-04-22T07:04:14.783345+0000","last_peered":"2024-04-22T07:04:14.783345+0000","last_clean":"2024-04-22T07:04:14.783345+0000","last_became_active":"2024-04-22T07:04:13.794793+0000","last_became_peered":"2024-04-22T07:04:13.794793+0000","last_unstale":"2024-04-22T07:04:14.783345+0000","last_undegraded":"2024-04-22T07:04:14.783345+0000","last_fullsized":"2024-04-22T07:04:14.783345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.879536+0000","last_change":"2024-04-22T07:04:12.789252+0000","last_active":"2024-04-22T07:04:39.879536+0000","last_peered":"2024-04-22T07:04:39.879536+0000","last_clean":"2024-04-22T07:04:39.879536+0000","last_became_active":"2024-04-22T07:04:12.788946+0000","last_became_peered":"2024-04-22T07:04:12.788946+0000","last_unstale":"2024-04-22T07:04:39.879536+0000","last_undegraded":"2024-04-22T07:04:39.879536+0000","last_fullsized":"2024-04-22T07:04:39.879536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:36:13.428703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961272+0000","last_change":"2024-04-22T07:04:13.801250+0000","last_active":"2024-04-22T07:04:14.961272+0000","last_peered":"2024-04-22T07:04:14.961272+0000","last_clean":"2024-04-22T07:04:14.961272+0000","last_became_active":"2024-04-22T07:04:13.799143+0000","last_became_peered":"2024-04-22T07:04:13.799143+0000","last_unstale":"2024-04-22T07:04:14.961272+0000","last_undegraded":"2024-04-22T07:04:14.961272+0000","last_fullsized":"2024-04-22T07:04:14.961272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960952+0000","last_change":"2024-04-22T07:04:03.688462+0000","last_active":"2024-04-22T07:04:14.960952+0000","last_peered":"2024-04-22T07:04:14.960952+0000","last_clean":"2024-04-22T07:04:14.960952+0000","last_became_active":"2024-04-22T07:04:03.688191+0000","last_became_peered":"2024-04-22T07:04:03.688191+0000","last_unstale":"2024-04-22T07:04:14.960952+0000","last_undegraded":"2024-04-22T07:04:14.960952+0000","last_fullsized":"2024-04-22T07:04:14.960952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.464330+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.053999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.877201+0000","last_change":"2024-04-22T07:04:12.783751+0000","last_active":"2024-04-22T07:04:39.877201+0000","last_peered":"2024-04-22T07:04:39.877201+0000","last_clean":"2024-04-22T07:04:39.877201+0000","last_became_active":"2024-04-22T07:04:12.783460+0000","last_became_peered":"2024-04-22T07:04:12.783460+0000","last_unstale":"2024-04-22T07:04:39.877201+0000","last_undegraded":"2024-04-22T07:04:39.877201+0000","last_fullsized":"2024-04-22T07:04:39.877201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054881+0000","last_change":"2024-04-22T07:04:13.791961+0000","last_active":"2024-04-22T07:04:15.054881+0000","last_peered":"2024-04-22T07:04:15.054881+0000","last_clean":"2024-04-22T07:04:15.054881+0000","last_became_active":"2024-04-22T07:04:13.790591+0000","last_became_peered":"2024-04-22T07:04:13.790591+0000","last_unstale":"2024-04-22T07:04:15.054881+0000","last_undegraded":"2024-04-22T07:04:15.054881+0000","last_fullsized":"2024-04-22T07:04:15.054881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:33.856152+0000","last_change":"2024-04-22T07:04:12.787268+0000","last_active":"2024-04-22T07:04:33.856152+0000","last_peered":"2024-04-22T07:04:33.856152+0000","last_clean":"2024-04-22T07:04:33.856152+0000","last_became_active":"2024-04-22T07:04:12.786991+0000","last_became_peered":"2024-04-22T07:04:12.786991+0000","last_unstale":"2024-04-22T07:04:33.856152+0000","last_undegraded":"2024-04-22T07:04:33.856152+0000","last_fullsized":"2024-04-22T07:04:33.856152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054726+0000","last_change":"2024-04-22T07:04:13.800243+0000","last_active":"2024-04-22T07:04:15.054726+0000","last_peered":"2024-04-22T07:04:15.054726+0000","last_clean":"2024-04-22T07:04:15.054726+0000","last_became_active":"2024-04-22T07:04:13.799917+0000","last_became_peered":"2024-04-22T07:04:13.799917+0000","last_unstale":"2024-04-22T07:04:15.054726+0000","last_undegraded":"2024-04-22T07:04:15.054726+0000","last_fullsized":"2024-04-22T07:04:15.054726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:54.958916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944501+0000","last_change":"2024-04-22T07:04:12.787893+0000","last_active":"2024-04-22T07:04:44.944501+0000","last_peered":"2024-04-22T07:04:44.944501+0000","last_clean":"2024-04-22T07:04:44.944501+0000","last_became_active":"2024-04-22T07:04:12.787370+0000","last_became_peered":"2024-04-22T07:04:12.787370+0000","last_unstale":"2024-04-22T07:04:44.944501+0000","last_undegraded":"2024-04-22T07:04:44.944501+0000","last_fullsized":"2024-04-22T07:04:44.944501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:52:03.220670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055952+0000","last_change":"2024-04-22T07:04:13.795454+0000","last_active":"2024-04-22T07:04:15.055952+0000","last_peered":"2024-04-22T07:04:15.055952+0000","last_clean":"2024-04-22T07:04:15.055952+0000","last_became_active":"2024-04-22T07:04:13.795138+0000","last_became_peered":"2024-04-22T07:04:13.795138+0000","last_unstale":"2024-04-22T07:04:15.055952+0000","last_undegraded":"2024-04-22T07:04:15.055952+0000","last_fullsized":"2024-04-22T07:04:15.055952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944953+0000","last_change":"2024-04-22T07:04:12.786891+0000","last_active":"2024-04-22T07:04:44.944953+0000","last_peered":"2024-04-22T07:04:44.944953+0000","last_clean":"2024-04-22T07:04:44.944953+0000","last_became_active":"2024-04-22T07:04:12.786557+0000","last_became_peered":"2024-04-22T07:04:12.786557+0000","last_unstale":"2024-04-22T07:04:44.944953+0000","last_undegraded":"2024-04-22T07:04:44.944953+0000","last_fullsized":"2024-04-22T07:04:44.944953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055054+0000","last_change":"2024-04-22T07:04:13.798678+0000","last_active":"2024-04-22T07:04:15.055054+0000","last_peered":"2024-04-22T07:04:15.055054+0000","last_clean":"2024-04-22T07:04:15.055054+0000","last_became_active":"2024-04-22T07:04:13.798369+0000","last_became_peered":"2024-04-22T07:04:13.798369+0000","last_unstale":"2024-04-22T07:04:15.055054+0000","last_undegraded":"2024-04-22T07:04:15.055054+0000","last_fullsized":"2024-04-22T07:04:15.055054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:37.919431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.731812+0000","last_change":"2024-04-22T07:04:12.791897+0000","last_active":"2024-04-22T07:04:15.731812+0000","last_peered":"2024-04-22T07:04:15.731812+0000","last_clean":"2024-04-22T07:04:15.731812+0000","last_became_active":"2024-04-22T07:04:12.791580+0000","last_became_peered":"2024-04-22T07:04:12.791580+0000","last_unstale":"2024-04-22T07:04:15.731812+0000","last_undegraded":"2024-04-22T07:04:15.731812+0000","last_fullsized":"2024-04-22T07:04:15.731812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782942+0000","last_change":"2024-04-22T07:04:13.803516+0000","last_active":"2024-04-22T07:04:14.782942+0000","last_peered":"2024-04-22T07:04:14.782942+0000","last_clean":"2024-04-22T07:04:14.782942+0000","last_became_active":"2024-04-22T07:04:13.803230+0000","last_became_peered":"2024-04-22T07:04:13.803230+0000","last_unstale":"2024-04-22T07:04:14.782942+0000","last_undegraded":"2024-04-22T07:04:14.782942+0000","last_fullsized":"2024-04-22T07:04:14.782942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.877324+0000","last_change":"2024-04-22T07:04:12.789499+0000","last_active":"2024-04-22T07:04:39.877324+0000","last_peered":"2024-04-22T07:04:39.877324+0000","last_clean":"2024-04-22T07:04:39.877324+0000","last_became_active":"2024-04-22T07:04:12.789229+0000","last_became_peered":"2024-04-22T07:04:12.789229+0000","last_unstale":"2024-04-22T07:04:39.877324+0000","last_undegraded":"2024-04-22T07:04:39.877324+0000","last_fullsized":"2024-04-22T07:04:39.877324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:53.955113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055856+0000","last_change":"2024-04-22T07:04:13.797969+0000","last_active":"2024-04-22T07:04:15.055856+0000","last_peered":"2024-04-22T07:04:15.055856+0000","last_clean":"2024-04-22T07:04:15.055856+0000","last_became_active":"2024-04-22T07:04:13.797662+0000","last_became_peered":"2024-04-22T07:04:13.797662+0000","last_unstale":"2024-04-22T07:04:15.055856+0000","last_undegraded":"2024-04-22T07:04:15.055856+0000","last_fullsized":"2024-04-22T07:04:15.055856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.874965+0000","last_change":"2024-04-22T07:04:12.791787+0000","last_active":"2024-04-22T07:04:39.874965+0000","last_peered":"2024-04-22T07:04:39.874965+0000","last_clean":"2024-04-22T07:04:39.874965+0000","last_became_active":"2024-04-22T07:04:12.791460+0000","last_became_peered":"2024-04-22T07:04:12.791460+0000","last_unstale":"2024-04-22T07:04:39.874965+0000","last_undegraded":"2024-04-22T07:04:39.874965+0000","last_fullsized":"2024-04-22T07:04:39.874965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055451+0000","last_change":"2024-04-22T07:04:13.797755+0000","last_active":"2024-04-22T07:04:15.055451+0000","last_peered":"2024-04-22T07:04:15.055451+0000","last_clean":"2024-04-22T07:04:15.055451+0000","last_became_active":"2024-04-22T07:04:13.797400+0000","last_became_peered":"2024-04-22T07:04:13.797400+0000","last_unstale":"2024-04-22T07:04:15.055451+0000","last_undegraded":"2024-04-22T07:04:15.055451+0000","last_fullsized":"2024-04-22T07:04:15.055451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945268+0000","last_change":"2024-04-22T07:04:12.790458+0000","last_active":"2024-04-22T07:04:44.945268+0000","last_peered":"2024-04-22T07:04:44.945268+0000","last_clean":"2024-04-22T07:04:44.945268+0000","last_became_active":"2024-04-22T07:04:12.790209+0000","last_became_peered":"2024-04-22T07:04:12.790209+0000","last_unstale":"2024-04-22T07:04:44.945268+0000","last_undegraded":"2024-04-22T07:04:44.945268+0000","last_fullsized":"2024-04-22T07:04:44.945268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056058+0000","last_change":"2024-04-22T07:04:13.796176+0000","last_active":"2024-04-22T07:04:15.056058+0000","last_peered":"2024-04-22T07:04:15.056058+0000","last_clean":"2024-04-22T07:04:15.056058+0000","last_became_active":"2024-04-22T07:04:13.795325+0000","last_became_peered":"2024-04-22T07:04:13.795325+0000","last_unstale":"2024-04-22T07:04:15.056058+0000","last_undegraded":"2024-04-22T07:04:15.056058+0000","last_fullsized":"2024-04-22T07:04:15.056058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.832746+0000","last_change":"2024-04-22T07:04:12.791157+0000","last_active":"2024-04-22T07:04:27.832746+0000","last_peered":"2024-04-22T07:04:27.832746+0000","last_clean":"2024-04-22T07:04:27.832746+0000","last_became_active":"2024-04-22T07:04:12.790879+0000","last_became_peered":"2024-04-22T07:04:12.790879+0000","last_unstale":"2024-04-22T07:04:27.832746+0000","last_undegraded":"2024-04-22T07:04:27.832746+0000","last_fullsized":"2024-04-22T07:04:27.832746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054928+0000","last_change":"2024-04-22T07:04:13.796587+0000","last_active":"2024-04-22T07:04:15.054928+0000","last_peered":"2024-04-22T07:04:15.054928+0000","last_clean":"2024-04-22T07:04:15.054928+0000","last_became_active":"2024-04-22T07:04:13.796305+0000","last_became_peered":"2024-04-22T07:04:13.796305+0000","last_unstale":"2024-04-22T07:04:15.054928+0000","last_undegraded":"2024-04-22T07:04:15.054928+0000","last_fullsized":"2024-04-22T07:04:15.054928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:29.793237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945446+0000","last_change":"2024-04-22T07:04:12.784359+0000","last_active":"2024-04-22T07:04:44.945446+0000","last_peered":"2024-04-22T07:04:44.945446+0000","last_clean":"2024-04-22T07:04:44.945446+0000","last_became_active":"2024-04-22T07:04:12.784076+0000","last_became_peered":"2024-04-22T07:04:12.784076+0000","last_unstale":"2024-04-22T07:04:44.945446+0000","last_undegraded":"2024-04-22T07:04:44.945446+0000","last_fullsized":"2024-04-22T07:04:44.945446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054875+0000","last_change":"2024-04-22T07:04:13.802658+0000","last_active":"2024-04-22T07:04:15.054875+0000","last_peered":"2024-04-22T07:04:15.054875+0000","last_clean":"2024-04-22T07:04:15.054875+0000","last_became_active":"2024-04-22T07:04:13.802059+0000","last_became_peered":"2024-04-22T07:04:13.802059+0000","last_unstale":"2024-04-22T07:04:15.054875+0000","last_undegraded":"2024-04-22T07:04:15.054875+0000","last_fullsized":"2024-04-22T07:04:15.054875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.874697+0000","last_change":"2024-04-22T07:04:12.788090+0000","last_active":"2024-04-22T07:04:39.874697+0000","last_peered":"2024-04-22T07:04:39.874697+0000","last_clean":"2024-04-22T07:04:39.874697+0000","last_became_active":"2024-04-22T07:04:12.787818+0000","last_became_peered":"2024-04-22T07:04:12.787818+0000","last_unstale":"2024-04-22T07:04:39.874697+0000","last_undegraded":"2024-04-22T07:04:39.874697+0000","last_fullsized":"2024-04-22T07:04:39.874697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055442+0000","last_change":"2024-04-22T07:04:13.802013+0000","last_active":"2024-04-22T07:04:15.055442+0000","last_peered":"2024-04-22T07:04:15.055442+0000","last_clean":"2024-04-22T07:04:15.055442+0000","last_became_active":"2024-04-22T07:04:13.801404+0000","last_became_peered":"2024-04-22T07:04:13.801404+0000","last_unstale":"2024-04-22T07:04:15.055442+0000","last_undegraded":"2024-04-22T07:04:15.055442+0000","last_fullsized":"2024-04-22T07:04:15.055442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.830917+0000","last_change":"2024-04-22T07:04:12.790981+0000","last_active":"2024-04-22T07:04:27.830917+0000","last_peered":"2024-04-22T07:04:27.830917+0000","last_clean":"2024-04-22T07:04:27.830917+0000","last_became_active":"2024-04-22T07:04:12.790592+0000","last_became_peered":"2024-04-22T07:04:12.790592+0000","last_unstale":"2024-04-22T07:04:27.830917+0000","last_undegraded":"2024-04-22T07:04:27.830917+0000","last_fullsized":"2024-04-22T07:04:27.830917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:53.115405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781878+0000","last_change":"2024-04-22T07:04:13.800619+0000","last_active":"2024-04-22T07:04:14.781878+0000","last_peered":"2024-04-22T07:04:14.781878+0000","last_clean":"2024-04-22T07:04:14.781878+0000","last_became_active":"2024-04-22T07:04:13.800279+0000","last_became_peered":"2024-04-22T07:04:13.800279+0000","last_unstale":"2024-04-22T07:04:14.781878+0000","last_undegraded":"2024-04-22T07:04:14.781878+0000","last_fullsized":"2024-04-22T07:04:14.781878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782949+0000","last_change":"2024-04-22T07:04:13.789060+0000","last_active":"2024-04-22T07:04:14.782949+0000","last_peered":"2024-04-22T07:04:14.782949+0000","last_clean":"2024-04-22T07:04:14.782949+0000","last_became_active":"2024-04-22T07:04:13.788787+0000","last_became_peered":"2024-04-22T07:04:13.788787+0000","last_unstale":"2024-04-22T07:04:14.782949+0000","last_undegraded":"2024-04-22T07:04:14.782949+0000","last_fullsized":"2024-04-22T07:04:14.782949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944673+0000","last_change":"2024-04-22T07:04:12.785077+0000","last_active":"2024-04-22T07:04:44.944673+0000","last_peered":"2024-04-22T07:04:44.944673+0000","last_clean":"2024-04-22T07:04:44.944673+0000","last_became_active":"2024-04-22T07:04:12.784804+0000","last_became_peered":"2024-04-22T07:04:12.784804+0000","last_unstale":"2024-04-22T07:04:44.944673+0000","last_undegraded":"2024-04-22T07:04:44.944673+0000","last_fullsized":"2024-04-22T07:04:44.944673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783259+0000","last_change":"2024-04-22T07:04:13.789866+0000","last_active":"2024-04-22T07:04:14.783259+0000","last_peered":"2024-04-22T07:04:14.783259+0000","last_clean":"2024-04-22T07:04:14.783259+0000","last_became_active":"2024-04-22T07:04:13.789539+0000","last_became_peered":"2024-04-22T07:04:13.789539+0000","last_unstale":"2024-04-22T07:04:14.783259+0000","last_undegraded":"2024-04-22T07:04:14.783259+0000","last_fullsized":"2024-04-22T07:04:14.783259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'12","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945287+0000","last_change":"2024-04-22T07:04:12.786162+0000","last_active":"2024-04-22T07:04:44.945287+0000","last_peered":"2024-04-22T07:04:44.945287+0000","last_clean":"2024-04-22T07:04:44.945287+0000","last_became_active":"2024-04-22T07:04:12.785310+0000","last_became_peered":"2024-04-22T07:04:12.785310+0000","last_unstale":"2024-04-22T07:04:44.945287+0000","last_undegraded":"2024-04-22T07:04:44.945287+0000","last_fullsized":"2024-04-22T07:04:44.945287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:06.776205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056634+0000","last_change":"2024-04-22T07:04:13.796267+0000","last_active":"2024-04-22T07:04:15.056634+0000","last_peered":"2024-04-22T07:04:15.056634+0000","last_clean":"2024-04-22T07:04:15.056634+0000","last_became_active":"2024-04-22T07:04:13.795220+0000","last_became_peered":"2024-04-22T07:04:13.795220+0000","last_unstale":"2024-04-22T07:04:15.056634+0000","last_undegraded":"2024-04-22T07:04:15.056634+0000","last_fullsized":"2024-04-22T07:04:15.056634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.802121+0000","last_change":"2024-04-22T07:04:12.785962+0000","last_active":"2024-04-22T07:04:18.802121+0000","last_peered":"2024-04-22T07:04:18.802121+0000","last_clean":"2024-04-22T07:04:18.802121+0000","last_became_active":"2024-04-22T07:04:12.785608+0000","last_became_peered":"2024-04-22T07:04:12.785608+0000","last_unstale":"2024-04-22T07:04:18.802121+0000","last_undegraded":"2024-04-22T07:04:18.802121+0000","last_fullsized":"2024-04-22T07:04:18.802121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055613+0000","last_change":"2024-04-22T07:04:13.801662+0000","last_active":"2024-04-22T07:04:15.055613+0000","last_peered":"2024-04-22T07:04:15.055613+0000","last_clean":"2024-04-22T07:04:15.055613+0000","last_became_active":"2024-04-22T07:04:13.798282+0000","last_became_peered":"2024-04-22T07:04:13.798282+0000","last_unstale":"2024-04-22T07:04:15.055613+0000","last_undegraded":"2024-04-22T07:04:15.055613+0000","last_fullsized":"2024-04-22T07:04:15.055613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'13","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945261+0000","last_change":"2024-04-22T07:04:12.791445+0000","last_active":"2024-04-22T07:04:44.945261+0000","last_peered":"2024-04-22T07:04:44.945261+0000","last_clean":"2024-04-22T07:04:44.945261+0000","last_became_active":"2024-04-22T07:04:12.791150+0000","last_became_peered":"2024-04-22T07:04:12.791150+0000","last_unstale":"2024-04-22T07:04:44.945261+0000","last_undegraded":"2024-04-22T07:04:44.945261+0000","last_fullsized":"2024-04-22T07:04:44.945261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8475,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":207,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":864256,"data_stored":673703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":83637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":85167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":258048,"data_stored":82565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":83671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":82767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":856064,"data_stored":672477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":83138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:55.764 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.902+0000 7f9a982fd640 1 Processor -- start 2024-04-22T07:04:55.900 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.902+0000 7f9a982fd640 1 -- start start 2024-04-22T07:04:55.901 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.903+0000 7f9a982fd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9a90152e00 0x7f9a9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.903+0000 7f9a982fd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a90159580 0x7f9a900c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.903+0000 7f9a982fd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a900c7ec0 con 0x7f9a900c6f70 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.903+0000 7f9a982fd640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f9a90071210 con 0x7f9a90152e00 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.903+0000 7f9a982fd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f9a901531e0 con 0x7f9a90159580 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.903+0000 7f9a96099640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9a90152e00 0x7f9a9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.903+0000 7f9a96099640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f9a90152e00 0x7f9a9006fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36762/0 (socket says 172.21.15.44:36762) 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.903+0000 7f9a96099640 1 -- 172.21.15.44:0/526777467 learned_addr learned my addr 172.21.15.44:0/526777467 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.903+0000 7f9a95898640 1 --2- 172.21.15.44:0/526777467 >> v2:172.21.15.70:3300/0 conn(0x7f9a90159580 0x7f9a900c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.904+0000 7f9a96099640 1 -- 172.21.15.44:0/526777467 >> v2:172.21.15.70:3300/0 conn(0x7f9a90159580 msgr2=0x7f9a900c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.904+0000 7f9a96099640 1 --2- 172.21.15.44:0/526777467 >> v2:172.21.15.70:3300/0 conn(0x7f9a90159580 0x7f9a900c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.904+0000 7f9a96099640 1 -- 172.21.15.44:0/526777467 >> v1:172.21.15.70:6789/0 conn(0x7f9a900c6f70 legacy=0x7f9a9015c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.904+0000 7f9a96099640 1 -- 172.21.15.44:0/526777467 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a90070c50 con 0x7f9a90152e00 2024-04-22T07:04:55.902 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.904+0000 7f9a95898640 1 --2- 172.21.15.44:0/526777467 >> v2:172.21.15.70:3300/0 conn(0x7f9a90159580 0x7f9a900c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.904+0000 7f9a96099640 1 --2- 172.21.15.44:0/526777467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a90152e00 0x7f9a9006fd50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9a9006e890 tx=0x7f9a8802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9858d1ab7d30a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.904+0000 7f9a95097640 1 -- 172.21.15.44:0/526777467 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8802ca80 con 0x7f9a90152e00 2024-04-22T07:04:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.904+0000 7f9a95097640 1 -- 172.21.15.44:0/526777467 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8802cc20 con 0x7f9a90152e00 2024-04-22T07:04:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.904+0000 7f9a95097640 1 -- 172.21.15.44:0/526777467 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a880343f0 con 0x7f9a90152e00 2024-04-22T07:04:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.905+0000 7f9a982fd640 1 -- 172.21.15.44:0/526777467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a90152e00 msgr2=0x7f9a9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.905+0000 7f9a982fd640 1 --2- 172.21.15.44:0/526777467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a90152e00 0x7f9a9006fd50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9a9006e890 tx=0x7f9a8802bee0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.905+0000 7f9a982fd640 1 -- 172.21.15.44:0/526777467 shutdown_connections 2024-04-22T07:04:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.905+0000 7f9a982fd640 1 --2- 172.21.15.44:0/526777467 >> v2:172.21.15.70:3300/0 conn(0x7f9a90159580 0x7f9a900c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.905+0000 7f9a982fd640 1 --2- 172.21.15.44:0/526777467 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a90152e00 0x7f9a9006fd50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.904 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.905+0000 7f9a982fd640 1 -- 172.21.15.44:0/526777467 >> 172.21.15.44:0/526777467 conn(0x7f9a900cd320 msgr2=0x7f9a900cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:55.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.905+0000 7f9a982fd640 1 -- 172.21.15.44:0/526777467 shutdown_connections 2024-04-22T07:04:55.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.905+0000 7f9a982fd640 1 -- 172.21.15.44:0/526777467 wait complete. 2024-04-22T07:04:55.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.906+0000 7f9a982fd640 1 Processor -- start 2024-04-22T07:04:55.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.906+0000 7f9a982fd640 1 -- start start 2024-04-22T07:04:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a982fd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a900c6f70 0x7f9a900cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a982fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a90152e00 0x7f9a901511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a982fd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a90159580 0x7f9a901f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a982fd640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f9a901599f0 con 0x7f9a900c6f70 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a982fd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9a900c8160 con 0x7f9a90152e00 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a982fd640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f9a90070530 con 0x7f9a90159580 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a96099640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a900c6f70 0x7f9a900cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a96099640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a900c6f70 0x7f9a900cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36770/0 (socket says 172.21.15.44:36770) 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a96099640 1 -- 172.21.15.44:0/4031193143 learned_addr learned my addr 172.21.15.44:0/4031193143 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a96099640 1 -- 172.21.15.44:0/4031193143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a90159580 msgr2=0x7f9a901f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a96099640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a90159580 0x7f9a901f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a96099640 1 -- 172.21.15.44:0/4031193143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a90152e00 msgr2=0x7f9a901511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a95898640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a90152e00 0x7f9a901511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a96099640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a90152e00 0x7f9a901511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.907+0000 7f9a96099640 1 -- 172.21.15.44:0/4031193143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a900cb660 con 0x7f9a900c6f70 2024-04-22T07:04:55.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.908+0000 7f9a96099640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a900c6f70 0x7f9a900cb120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9a88036040 tx=0x7f9a88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.908+0000 7f9a8effd640 1 -- 172.21.15.44:0/4031193143 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88043020 con 0x7f9a900c6f70 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.908+0000 7f9a8effd640 1 -- 172.21.15.44:0/4031193143 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8803e450 con 0x7f9a900c6f70 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.908+0000 7f9a982fd640 1 -- 172.21.15.44:0/4031193143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9014c8f0 con 0x7f9a900c6f70 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.908+0000 7f9a8effd640 1 -- 172.21.15.44:0/4031193143 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88009510 con 0x7f9a900c6f70 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.908+0000 7f9a982fd640 1 -- 172.21.15.44:0/4031193143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a901eeaa0 con 0x7f9a900c6f70 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.909+0000 7f9a982fd640 1 -- 172.21.15.44:0/4031193143 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a9004fdc0 con 0x7f9a900c6f70 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.909+0000 7f9a8effd640 1 -- 172.21.15.44:0/4031193143 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a88046020 con 0x7f9a900c6f70 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.910+0000 7f9a8effd640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9a7c06c8f0 0x7f9a7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.910+0000 7f9a8effd640 1 -- 172.21.15.44:0/4031193143 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f9a880ae420 con 0x7f9a900c6f70 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.910+0000 7f9a95898640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9a7c06c8f0 0x7f9a7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:55.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.910+0000 7f9a95898640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9a7c06c8f0 0x7f9a7c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9a90069230 tx=0x7f9a84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:55.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:55.914+0000 7f9a8effd640 1 -- 172.21.15.44:0/4031193143 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a8804d030 con 0x7f9a900c6f70 2024-04-22T07:04:56.099 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.099+0000 7f9a982fd640 1 -- 172.21.15.44:0/4031193143 --> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a9014ceb0 con 0x7f9a7c06c8f0 2024-04-22T07:04:56.103 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.105+0000 7f9a8effd640 1 -- 172.21.15.44:0/4031193143 <== mgr.4101 v2:172.21.15.44:6832/1889438723 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251920 (secure 0 0 0) 0x7f9a9014ceb0 con 0x7f9a7c06c8f0 2024-04-22T07:04:56.106 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:56.108 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T07:04:56.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 -- 172.21.15.44:0/4031193143 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9a7c06c8f0 msgr2=0x7f9a7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9a7c06c8f0 0x7f9a7c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9a90069230 tx=0x7f9a84008040 comp rx=0 tx=0).stop 2024-04-22T07:04:56.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 -- 172.21.15.44:0/4031193143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a900c6f70 msgr2=0x7f9a900cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a900c6f70 0x7f9a900cb120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9a88036040 tx=0x7f9a88004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:56.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 -- 172.21.15.44:0/4031193143 shutdown_connections 2024-04-22T07:04:56.110 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f9a7c06c8f0 0x7f9a7c06edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f9a90159580 0x7f9a901f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a90152e00 0x7f9a901511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 --2- 172.21.15.44:0/4031193143 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f9a900c6f70 0x7f9a900cb120 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 -- 172.21.15.44:0/4031193143 >> 172.21.15.44:0/4031193143 conn(0x7f9a900cd320 msgr2=0x7f9a900c36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:56.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 -- 172.21.15.44:0/4031193143 shutdown_connections 2024-04-22T07:04:56.111 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.108+0000 7f9a982fd640 1 -- 172.21.15.44:0/4031193143 wait complete. 2024-04-22T07:04:56.116 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-04-22T07:04:55.721228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598843,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":366,"num_write_kb":2480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240168,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734552,"statfs":{"total":773094113280,"available":772848181248,"internally_reserved":0,"allocated":3317760,"data_stored":1847125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":463,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001790"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961498+0000","last_change":"2024-04-22T07:04:13.800864+0000","last_active":"2024-04-22T07:04:14.961498+0000","last_peered":"2024-04-22T07:04:14.961498+0000","last_clean":"2024-04-22T07:04:14.961498+0000","last_became_active":"2024-04-22T07:04:13.800611+0000","last_became_peered":"2024-04-22T07:04:13.800611+0000","last_unstale":"2024-04-22T07:04:14.961498+0000","last_undegraded":"2024-04-22T07:04:14.961498+0000","last_fullsized":"2024-04-22T07:04:14.961498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:55.129127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056077+0000","last_change":"2024-04-22T07:04:13.798390+0000","last_active":"2024-04-22T07:04:15.056077+0000","last_peered":"2024-04-22T07:04:15.056077+0000","last_clean":"2024-04-22T07:04:15.056077+0000","last_became_active":"2024-04-22T07:04:13.797927+0000","last_became_peered":"2024-04-22T07:04:13.797927+0000","last_unstale":"2024-04-22T07:04:15.056077+0000","last_undegraded":"2024-04-22T07:04:15.056077+0000","last_fullsized":"2024-04-22T07:04:15.056077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056425+0000","last_change":"2024-04-22T07:04:13.795288+0000","last_active":"2024-04-22T07:04:15.056425+0000","last_peered":"2024-04-22T07:04:15.056425+0000","last_clean":"2024-04-22T07:04:15.056425+0000","last_became_active":"2024-04-22T07:04:13.794704+0000","last_became_peered":"2024-04-22T07:04:13.794704+0000","last_unstale":"2024-04-22T07:04:15.056425+0000","last_undegraded":"2024-04-22T07:04:15.056425+0000","last_fullsized":"2024-04-22T07:04:15.056425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783223+0000","last_change":"2024-04-22T07:04:13.801884+0000","last_active":"2024-04-22T07:04:14.783223+0000","last_peered":"2024-04-22T07:04:14.783223+0000","last_clean":"2024-04-22T07:04:14.783223+0000","last_became_active":"2024-04-22T07:04:13.801620+0000","last_became_peered":"2024-04-22T07:04:13.801620+0000","last_unstale":"2024-04-22T07:04:14.783223+0000","last_undegraded":"2024-04-22T07:04:14.783223+0000","last_fullsized":"2024-04-22T07:04:14.783223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961151+0000","last_change":"2024-04-22T07:04:13.798670+0000","last_active":"2024-04-22T07:04:14.961151+0000","last_peered":"2024-04-22T07:04:14.961151+0000","last_clean":"2024-04-22T07:04:14.961151+0000","last_became_active":"2024-04-22T07:04:13.798384+0000","last_became_peered":"2024-04-22T07:04:13.798384+0000","last_unstale":"2024-04-22T07:04:14.961151+0000","last_undegraded":"2024-04-22T07:04:14.961151+0000","last_fullsized":"2024-04-22T07:04:14.961151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783124+0000","last_change":"2024-04-22T07:04:13.796468+0000","last_active":"2024-04-22T07:04:14.783124+0000","last_peered":"2024-04-22T07:04:14.783124+0000","last_clean":"2024-04-22T07:04:14.783124+0000","last_became_active":"2024-04-22T07:04:13.796139+0000","last_became_peered":"2024-04-22T07:04:13.796139+0000","last_unstale":"2024-04-22T07:04:14.783124+0000","last_undegraded":"2024-04-22T07:04:14.783124+0000","last_fullsized":"2024-04-22T07:04:14.783124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961066+0000","last_change":"2024-04-22T07:04:13.801449+0000","last_active":"2024-04-22T07:04:14.961066+0000","last_peered":"2024-04-22T07:04:14.961066+0000","last_clean":"2024-04-22T07:04:14.961066+0000","last_became_active":"2024-04-22T07:04:13.799374+0000","last_became_peered":"2024-04-22T07:04:13.799374+0000","last_unstale":"2024-04-22T07:04:14.961066+0000","last_undegraded":"2024-04-22T07:04:14.961066+0000","last_fullsized":"2024-04-22T07:04:14.961066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054559+0000","last_change":"2024-04-22T07:04:13.802780+0000","last_active":"2024-04-22T07:04:15.054559+0000","last_peered":"2024-04-22T07:04:15.054559+0000","last_clean":"2024-04-22T07:04:15.054559+0000","last_became_active":"2024-04-22T07:04:13.802371+0000","last_became_peered":"2024-04-22T07:04:13.802371+0000","last_unstale":"2024-04-22T07:04:15.054559+0000","last_undegraded":"2024-04-22T07:04:15.054559+0000","last_fullsized":"2024-04-22T07:04:15.054559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961532+0000","last_change":"2024-04-22T07:04:13.792483+0000","last_active":"2024-04-22T07:04:14.961532+0000","last_peered":"2024-04-22T07:04:14.961532+0000","last_clean":"2024-04-22T07:04:14.961532+0000","last_became_active":"2024-04-22T07:04:13.792198+0000","last_became_peered":"2024-04-22T07:04:13.792198+0000","last_unstale":"2024-04-22T07:04:14.961532+0000","last_undegraded":"2024-04-22T07:04:14.961532+0000","last_fullsized":"2024-04-22T07:04:14.961532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:36.943428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782040+0000","last_change":"2024-04-22T07:04:13.797379+0000","last_active":"2024-04-22T07:04:14.782040+0000","last_peered":"2024-04-22T07:04:14.782040+0000","last_clean":"2024-04-22T07:04:14.782040+0000","last_became_active":"2024-04-22T07:04:13.797029+0000","last_became_peered":"2024-04-22T07:04:13.797029+0000","last_unstale":"2024-04-22T07:04:14.782040+0000","last_undegraded":"2024-04-22T07:04:14.782040+0000","last_fullsized":"2024-04-22T07:04:14.782040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781680+0000","last_change":"2024-04-22T07:04:13.789875+0000","last_active":"2024-04-22T07:04:14.781680+0000","last_peered":"2024-04-22T07:04:14.781680+0000","last_clean":"2024-04-22T07:04:14.781680+0000","last_became_active":"2024-04-22T07:04:13.789598+0000","last_became_peered":"2024-04-22T07:04:13.789598+0000","last_unstale":"2024-04-22T07:04:14.781680+0000","last_undegraded":"2024-04-22T07:04:14.781680+0000","last_fullsized":"2024-04-22T07:04:14.781680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055156+0000","last_change":"2024-04-22T07:04:13.791103+0000","last_active":"2024-04-22T07:04:15.055156+0000","last_peered":"2024-04-22T07:04:15.055156+0000","last_clean":"2024-04-22T07:04:15.055156+0000","last_became_active":"2024-04-22T07:04:13.790819+0000","last_became_peered":"2024-04-22T07:04:13.790819+0000","last_unstale":"2024-04-22T07:04:15.055156+0000","last_undegraded":"2024-04-22T07:04:15.055156+0000","last_fullsized":"2024-04-22T07:04:15.055156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:50.632020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055402+0000","last_change":"2024-04-22T07:04:13.802407+0000","last_active":"2024-04-22T07:04:15.055402+0000","last_peered":"2024-04-22T07:04:15.055402+0000","last_clean":"2024-04-22T07:04:15.055402+0000","last_became_active":"2024-04-22T07:04:13.801785+0000","last_became_peered":"2024-04-22T07:04:13.801785+0000","last_unstale":"2024-04-22T07:04:15.055402+0000","last_undegraded":"2024-04-22T07:04:15.055402+0000","last_fullsized":"2024-04-22T07:04:15.055402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055557+0000","last_change":"2024-04-22T07:04:13.795888+0000","last_active":"2024-04-22T07:04:15.055557+0000","last_peered":"2024-04-22T07:04:15.055557+0000","last_clean":"2024-04-22T07:04:15.055557+0000","last_became_active":"2024-04-22T07:04:13.795092+0000","last_became_peered":"2024-04-22T07:04:13.795092+0000","last_unstale":"2024-04-22T07:04:15.055557+0000","last_undegraded":"2024-04-22T07:04:15.055557+0000","last_fullsized":"2024-04-22T07:04:15.055557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:54.119208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055926+0000","last_change":"2024-04-22T07:04:13.797138+0000","last_active":"2024-04-22T07:04:15.055926+0000","last_peered":"2024-04-22T07:04:15.055926+0000","last_clean":"2024-04-22T07:04:15.055926+0000","last_became_active":"2024-04-22T07:04:13.796763+0000","last_became_peered":"2024-04-22T07:04:13.796763+0000","last_unstale":"2024-04-22T07:04:15.055926+0000","last_undegraded":"2024-04-22T07:04:15.055926+0000","last_fullsized":"2024-04-22T07:04:15.055926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782465+0000","last_change":"2024-04-22T07:04:13.803869+0000","last_active":"2024-04-22T07:04:14.782465+0000","last_peered":"2024-04-22T07:04:14.782465+0000","last_clean":"2024-04-22T07:04:14.782465+0000","last_became_active":"2024-04-22T07:04:13.803451+0000","last_became_peered":"2024-04-22T07:04:13.803451+0000","last_unstale":"2024-04-22T07:04:14.782465+0000","last_undegraded":"2024-04-22T07:04:14.782465+0000","last_fullsized":"2024-04-22T07:04:14.782465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783077+0000","last_change":"2024-04-22T07:04:13.795746+0000","last_active":"2024-04-22T07:04:14.783077+0000","last_peered":"2024-04-22T07:04:14.783077+0000","last_clean":"2024-04-22T07:04:14.783077+0000","last_became_active":"2024-04-22T07:04:13.794694+0000","last_became_peered":"2024-04-22T07:04:13.794694+0000","last_unstale":"2024-04-22T07:04:14.783077+0000","last_undegraded":"2024-04-22T07:04:14.783077+0000","last_fullsized":"2024-04-22T07:04:14.783077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054829+0000","last_change":"2024-04-22T07:04:13.790670+0000","last_active":"2024-04-22T07:04:15.054829+0000","last_peered":"2024-04-22T07:04:15.054829+0000","last_clean":"2024-04-22T07:04:15.054829+0000","last_became_active":"2024-04-22T07:04:13.790104+0000","last_became_peered":"2024-04-22T07:04:13.790104+0000","last_unstale":"2024-04-22T07:04:15.054829+0000","last_undegraded":"2024-04-22T07:04:15.054829+0000","last_fullsized":"2024-04-22T07:04:15.054829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:09.325971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961464+0000","last_change":"2024-04-22T07:04:13.799016+0000","last_active":"2024-04-22T07:04:14.961464+0000","last_peered":"2024-04-22T07:04:14.961464+0000","last_clean":"2024-04-22T07:04:14.961464+0000","last_became_active":"2024-04-22T07:04:13.798716+0000","last_became_peered":"2024-04-22T07:04:13.798716+0000","last_unstale":"2024-04-22T07:04:14.961464+0000","last_undegraded":"2024-04-22T07:04:14.961464+0000","last_fullsized":"2024-04-22T07:04:14.961464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:36.609382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781586+0000","last_change":"2024-04-22T07:04:13.797296+0000","last_active":"2024-04-22T07:04:14.781586+0000","last_peered":"2024-04-22T07:04:14.781586+0000","last_clean":"2024-04-22T07:04:14.781586+0000","last_became_active":"2024-04-22T07:04:13.796997+0000","last_became_peered":"2024-04-22T07:04:13.796997+0000","last_unstale":"2024-04-22T07:04:14.781586+0000","last_undegraded":"2024-04-22T07:04:14.781586+0000","last_fullsized":"2024-04-22T07:04:14.781586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055994+0000","last_change":"2024-04-22T07:04:13.799675+0000","last_active":"2024-04-22T07:04:15.055994+0000","last_peered":"2024-04-22T07:04:15.055994+0000","last_clean":"2024-04-22T07:04:15.055994+0000","last_became_active":"2024-04-22T07:04:13.799411+0000","last_became_peered":"2024-04-22T07:04:13.799411+0000","last_unstale":"2024-04-22T07:04:15.055994+0000","last_undegraded":"2024-04-22T07:04:15.055994+0000","last_fullsized":"2024-04-22T07:04:15.055994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783058+0000","last_change":"2024-04-22T07:04:13.796128+0000","last_active":"2024-04-22T07:04:14.783058+0000","last_peered":"2024-04-22T07:04:14.783058+0000","last_clean":"2024-04-22T07:04:14.783058+0000","last_became_active":"2024-04-22T07:04:13.795330+0000","last_became_peered":"2024-04-22T07:04:13.795330+0000","last_unstale":"2024-04-22T07:04:14.783058+0000","last_undegraded":"2024-04-22T07:04:14.783058+0000","last_fullsized":"2024-04-22T07:04:14.783058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055294+0000","last_change":"2024-04-22T07:04:13.800458+0000","last_active":"2024-04-22T07:04:15.055294+0000","last_peered":"2024-04-22T07:04:15.055294+0000","last_clean":"2024-04-22T07:04:15.055294+0000","last_became_active":"2024-04-22T07:04:13.800171+0000","last_became_peered":"2024-04-22T07:04:13.800171+0000","last_unstale":"2024-04-22T07:04:15.055294+0000","last_undegraded":"2024-04-22T07:04:15.055294+0000","last_fullsized":"2024-04-22T07:04:15.055294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782518+0000","last_change":"2024-04-22T07:04:13.802472+0000","last_active":"2024-04-22T07:04:14.782518+0000","last_peered":"2024-04-22T07:04:14.782518+0000","last_clean":"2024-04-22T07:04:14.782518+0000","last_became_active":"2024-04-22T07:04:13.802201+0000","last_became_peered":"2024-04-22T07:04:13.802201+0000","last_unstale":"2024-04-22T07:04:14.782518+0000","last_undegraded":"2024-04-22T07:04:14.782518+0000","last_fullsized":"2024-04-22T07:04:14.782518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055335+0000","last_change":"2024-04-22T07:04:13.795055+0000","last_active":"2024-04-22T07:04:15.055335+0000","last_peered":"2024-04-22T07:04:15.055335+0000","last_clean":"2024-04-22T07:04:15.055335+0000","last_became_active":"2024-04-22T07:04:13.794145+0000","last_became_peered":"2024-04-22T07:04:13.794145+0000","last_unstale":"2024-04-22T07:04:15.055335+0000","last_undegraded":"2024-04-22T07:04:15.055335+0000","last_fullsized":"2024-04-22T07:04:15.055335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782104+0000","last_change":"2024-04-22T07:04:13.799772+0000","last_active":"2024-04-22T07:04:14.782104+0000","last_peered":"2024-04-22T07:04:14.782104+0000","last_clean":"2024-04-22T07:04:14.782104+0000","last_became_active":"2024-04-22T07:04:13.799504+0000","last_became_peered":"2024-04-22T07:04:13.799504+0000","last_unstale":"2024-04-22T07:04:14.782104+0000","last_undegraded":"2024-04-22T07:04:14.782104+0000","last_fullsized":"2024-04-22T07:04:14.782104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056133+0000","last_change":"2024-04-22T07:04:13.804604+0000","last_active":"2024-04-22T07:04:15.056133+0000","last_peered":"2024-04-22T07:04:15.056133+0000","last_clean":"2024-04-22T07:04:15.056133+0000","last_became_active":"2024-04-22T07:04:13.803594+0000","last_became_peered":"2024-04-22T07:04:13.803594+0000","last_unstale":"2024-04-22T07:04:15.056133+0000","last_undegraded":"2024-04-22T07:04:15.056133+0000","last_fullsized":"2024-04-22T07:04:15.056133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055262+0000","last_change":"2024-04-22T07:04:13.795947+0000","last_active":"2024-04-22T07:04:15.055262+0000","last_peered":"2024-04-22T07:04:15.055262+0000","last_clean":"2024-04-22T07:04:15.055262+0000","last_became_active":"2024-04-22T07:04:13.795223+0000","last_became_peered":"2024-04-22T07:04:13.795223+0000","last_unstale":"2024-04-22T07:04:15.055262+0000","last_undegraded":"2024-04-22T07:04:15.055262+0000","last_fullsized":"2024-04-22T07:04:15.055262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056195+0000","last_change":"2024-04-22T07:04:13.804225+0000","last_active":"2024-04-22T07:04:15.056195+0000","last_peered":"2024-04-22T07:04:15.056195+0000","last_clean":"2024-04-22T07:04:15.056195+0000","last_became_active":"2024-04-22T07:04:13.803920+0000","last_became_peered":"2024-04-22T07:04:13.803920+0000","last_unstale":"2024-04-22T07:04:15.056195+0000","last_undegraded":"2024-04-22T07:04:15.056195+0000","last_fullsized":"2024-04-22T07:04:15.056195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783228+0000","last_change":"2024-04-22T07:04:13.796357+0000","last_active":"2024-04-22T07:04:14.783228+0000","last_peered":"2024-04-22T07:04:14.783228+0000","last_clean":"2024-04-22T07:04:14.783228+0000","last_became_active":"2024-04-22T07:04:13.795784+0000","last_became_peered":"2024-04-22T07:04:13.795784+0000","last_unstale":"2024-04-22T07:04:14.783228+0000","last_undegraded":"2024-04-22T07:04:14.783228+0000","last_fullsized":"2024-04-22T07:04:14.783228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054855+0000","last_change":"2024-04-22T07:04:13.784337+0000","last_active":"2024-04-22T07:04:15.054855+0000","last_peered":"2024-04-22T07:04:15.054855+0000","last_clean":"2024-04-22T07:04:15.054855+0000","last_became_active":"2024-04-22T07:04:13.783914+0000","last_became_peered":"2024-04-22T07:04:13.783914+0000","last_unstale":"2024-04-22T07:04:15.054855+0000","last_undegraded":"2024-04-22T07:04:15.054855+0000","last_fullsized":"2024-04-22T07:04:15.054855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:43:41.211395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781515+0000","last_change":"2024-04-22T07:04:13.806969+0000","last_active":"2024-04-22T07:04:14.781515+0000","last_peered":"2024-04-22T07:04:14.781515+0000","last_clean":"2024-04-22T07:04:14.781515+0000","last_became_active":"2024-04-22T07:04:13.806715+0000","last_became_peered":"2024-04-22T07:04:13.806715+0000","last_unstale":"2024-04-22T07:04:14.781515+0000","last_undegraded":"2024-04-22T07:04:14.781515+0000","last_fullsized":"2024-04-22T07:04:14.781515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054895+0000","last_change":"2024-04-22T07:04:13.794896+0000","last_active":"2024-04-22T07:04:15.054895+0000","last_peered":"2024-04-22T07:04:15.054895+0000","last_clean":"2024-04-22T07:04:15.054895+0000","last_became_active":"2024-04-22T07:04:13.794605+0000","last_became_peered":"2024-04-22T07:04:13.794605+0000","last_unstale":"2024-04-22T07:04:15.054895+0000","last_undegraded":"2024-04-22T07:04:15.054895+0000","last_fullsized":"2024-04-22T07:04:15.054895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:02.613130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.836956+0000","last_change":"2024-04-22T07:04:12.786738+0000","last_active":"2024-04-22T07:04:27.836956+0000","last_peered":"2024-04-22T07:04:27.836956+0000","last_clean":"2024-04-22T07:04:27.836956+0000","last_became_active":"2024-04-22T07:04:12.786413+0000","last_became_peered":"2024-04-22T07:04:12.786413+0000","last_unstale":"2024-04-22T07:04:27.836956+0000","last_undegraded":"2024-04-22T07:04:27.836956+0000","last_fullsized":"2024-04-22T07:04:27.836956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056196+0000","last_change":"2024-04-22T07:04:13.800525+0000","last_active":"2024-04-22T07:04:15.056196+0000","last_peered":"2024-04-22T07:04:15.056196+0000","last_clean":"2024-04-22T07:04:15.056196+0000","last_became_active":"2024-04-22T07:04:13.800254+0000","last_became_peered":"2024-04-22T07:04:13.800254+0000","last_unstale":"2024-04-22T07:04:15.056196+0000","last_undegraded":"2024-04-22T07:04:15.056196+0000","last_fullsized":"2024-04-22T07:04:15.056196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.873809+0000","last_change":"2024-04-22T07:04:12.786726+0000","last_active":"2024-04-22T07:04:39.873809+0000","last_peered":"2024-04-22T07:04:39.873809+0000","last_clean":"2024-04-22T07:04:39.873809+0000","last_became_active":"2024-04-22T07:04:12.786444+0000","last_became_peered":"2024-04-22T07:04:12.786444+0000","last_unstale":"2024-04-22T07:04:39.873809+0000","last_undegraded":"2024-04-22T07:04:39.873809+0000","last_fullsized":"2024-04-22T07:04:39.873809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:33.855518+0000","last_change":"2024-04-22T07:04:12.787357+0000","last_active":"2024-04-22T07:04:33.855518+0000","last_peered":"2024-04-22T07:04:33.855518+0000","last_clean":"2024-04-22T07:04:33.855518+0000","last_became_active":"2024-04-22T07:04:12.787066+0000","last_became_peered":"2024-04-22T07:04:12.787066+0000","last_unstale":"2024-04-22T07:04:33.855518+0000","last_undegraded":"2024-04-22T07:04:33.855518+0000","last_fullsized":"2024-04-22T07:04:33.855518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054713+0000","last_change":"2024-04-22T07:04:13.807697+0000","last_active":"2024-04-22T07:04:15.054713+0000","last_peered":"2024-04-22T07:04:15.054713+0000","last_clean":"2024-04-22T07:04:15.054713+0000","last_became_active":"2024-04-22T07:04:13.807327+0000","last_became_peered":"2024-04-22T07:04:13.807327+0000","last_unstale":"2024-04-22T07:04:15.054713+0000","last_undegraded":"2024-04-22T07:04:15.054713+0000","last_fullsized":"2024-04-22T07:04:15.054713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.873858+0000","last_change":"2024-04-22T07:04:12.787783+0000","last_active":"2024-04-22T07:04:39.873858+0000","last_peered":"2024-04-22T07:04:39.873858+0000","last_clean":"2024-04-22T07:04:39.873858+0000","last_became_active":"2024-04-22T07:04:12.786765+0000","last_became_peered":"2024-04-22T07:04:12.786765+0000","last_unstale":"2024-04-22T07:04:39.873858+0000","last_undegraded":"2024-04-22T07:04:39.873858+0000","last_fullsized":"2024-04-22T07:04:39.873858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055740+0000","last_change":"2024-04-22T07:04:13.804931+0000","last_active":"2024-04-22T07:04:15.055740+0000","last_peered":"2024-04-22T07:04:15.055740+0000","last_clean":"2024-04-22T07:04:15.055740+0000","last_became_active":"2024-04-22T07:04:13.804629+0000","last_became_peered":"2024-04-22T07:04:13.804629+0000","last_unstale":"2024-04-22T07:04:15.055740+0000","last_undegraded":"2024-04-22T07:04:15.055740+0000","last_fullsized":"2024-04-22T07:04:15.055740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.834749+0000","last_change":"2024-04-22T07:04:12.789279+0000","last_active":"2024-04-22T07:04:27.834749+0000","last_peered":"2024-04-22T07:04:27.834749+0000","last_clean":"2024-04-22T07:04:27.834749+0000","last_became_active":"2024-04-22T07:04:12.787705+0000","last_became_peered":"2024-04-22T07:04:12.787705+0000","last_unstale":"2024-04-22T07:04:27.834749+0000","last_undegraded":"2024-04-22T07:04:27.834749+0000","last_fullsized":"2024-04-22T07:04:27.834749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782812+0000","last_change":"2024-04-22T07:04:13.794292+0000","last_active":"2024-04-22T07:04:14.782812+0000","last_peered":"2024-04-22T07:04:14.782812+0000","last_clean":"2024-04-22T07:04:14.782812+0000","last_became_active":"2024-04-22T07:04:13.793986+0000","last_became_peered":"2024-04-22T07:04:13.793986+0000","last_unstale":"2024-04-22T07:04:14.782812+0000","last_undegraded":"2024-04-22T07:04:14.782812+0000","last_fullsized":"2024-04-22T07:04:14.782812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944030+0000","last_change":"2024-04-22T07:04:12.787064+0000","last_active":"2024-04-22T07:04:44.944030+0000","last_peered":"2024-04-22T07:04:44.944030+0000","last_clean":"2024-04-22T07:04:44.944030+0000","last_became_active":"2024-04-22T07:04:12.786430+0000","last_became_peered":"2024-04-22T07:04:12.786430+0000","last_unstale":"2024-04-22T07:04:44.944030+0000","last_undegraded":"2024-04-22T07:04:44.944030+0000","last_fullsized":"2024-04-22T07:04:44.944030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056732+0000","last_change":"2024-04-22T07:04:13.801337+0000","last_active":"2024-04-22T07:04:15.056732+0000","last_peered":"2024-04-22T07:04:15.056732+0000","last_clean":"2024-04-22T07:04:15.056732+0000","last_became_active":"2024-04-22T07:04:13.800916+0000","last_became_peered":"2024-04-22T07:04:13.800916+0000","last_unstale":"2024-04-22T07:04:15.056732+0000","last_undegraded":"2024-04-22T07:04:15.056732+0000","last_fullsized":"2024-04-22T07:04:15.056732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.834615+0000","last_change":"2024-04-22T07:04:12.787741+0000","last_active":"2024-04-22T07:04:27.834615+0000","last_peered":"2024-04-22T07:04:27.834615+0000","last_clean":"2024-04-22T07:04:27.834615+0000","last_became_active":"2024-04-22T07:04:12.787405+0000","last_became_peered":"2024-04-22T07:04:12.787405+0000","last_unstale":"2024-04-22T07:04:27.834615+0000","last_undegraded":"2024-04-22T07:04:27.834615+0000","last_fullsized":"2024-04-22T07:04:27.834615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055184+0000","last_change":"2024-04-22T07:04:13.795502+0000","last_active":"2024-04-22T07:04:15.055184+0000","last_peered":"2024-04-22T07:04:15.055184+0000","last_clean":"2024-04-22T07:04:15.055184+0000","last_became_active":"2024-04-22T07:04:13.795199+0000","last_became_peered":"2024-04-22T07:04:13.795199+0000","last_unstale":"2024-04-22T07:04:15.055184+0000","last_undegraded":"2024-04-22T07:04:15.055184+0000","last_fullsized":"2024-04-22T07:04:15.055184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:07.780008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'10","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.943628+0000","last_change":"2024-04-22T07:04:12.788572+0000","last_active":"2024-04-22T07:04:44.943628+0000","last_peered":"2024-04-22T07:04:44.943628+0000","last_clean":"2024-04-22T07:04:44.943628+0000","last_became_active":"2024-04-22T07:04:12.788286+0000","last_became_peered":"2024-04-22T07:04:12.788286+0000","last_unstale":"2024-04-22T07:04:44.943628+0000","last_undegraded":"2024-04-22T07:04:44.943628+0000","last_fullsized":"2024-04-22T07:04:44.943628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961160+0000","last_change":"2024-04-22T07:04:13.804435+0000","last_active":"2024-04-22T07:04:14.961160+0000","last_peered":"2024-04-22T07:04:14.961160+0000","last_clean":"2024-04-22T07:04:14.961160+0000","last_became_active":"2024-04-22T07:04:13.804170+0000","last_became_peered":"2024-04-22T07:04:13.804170+0000","last_unstale":"2024-04-22T07:04:14.961160+0000","last_undegraded":"2024-04-22T07:04:14.961160+0000","last_fullsized":"2024-04-22T07:04:14.961160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'9","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944342+0000","last_change":"2024-04-22T07:04:12.789905+0000","last_active":"2024-04-22T07:04:44.944342+0000","last_peered":"2024-04-22T07:04:44.944342+0000","last_clean":"2024-04-22T07:04:44.944342+0000","last_became_active":"2024-04-22T07:04:12.789503+0000","last_became_peered":"2024-04-22T07:04:12.789503+0000","last_unstale":"2024-04-22T07:04:44.944342+0000","last_undegraded":"2024-04-22T07:04:44.944342+0000","last_fullsized":"2024-04-22T07:04:44.944342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055741+0000","last_change":"2024-04-22T07:04:13.801490+0000","last_active":"2024-04-22T07:04:15.055741+0000","last_peered":"2024-04-22T07:04:15.055741+0000","last_clean":"2024-04-22T07:04:15.055741+0000","last_became_active":"2024-04-22T07:04:13.799356+0000","last_became_peered":"2024-04-22T07:04:13.799356+0000","last_unstale":"2024-04-22T07:04:15.055741+0000","last_undegraded":"2024-04-22T07:04:15.055741+0000","last_fullsized":"2024-04-22T07:04:15.055741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:32:04.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.877037+0000","last_change":"2024-04-22T07:04:12.781089+0000","last_active":"2024-04-22T07:04:39.877037+0000","last_peered":"2024-04-22T07:04:39.877037+0000","last_clean":"2024-04-22T07:04:39.877037+0000","last_became_active":"2024-04-22T07:04:12.780827+0000","last_became_peered":"2024-04-22T07:04:12.780827+0000","last_unstale":"2024-04-22T07:04:39.877037+0000","last_undegraded":"2024-04-22T07:04:39.877037+0000","last_fullsized":"2024-04-22T07:04:39.877037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961202+0000","last_change":"2024-04-22T07:04:13.801375+0000","last_active":"2024-04-22T07:04:14.961202+0000","last_peered":"2024-04-22T07:04:14.961202+0000","last_clean":"2024-04-22T07:04:14.961202+0000","last_became_active":"2024-04-22T07:04:13.799488+0000","last_became_peered":"2024-04-22T07:04:13.799488+0000","last_unstale":"2024-04-22T07:04:14.961202+0000","last_undegraded":"2024-04-22T07:04:14.961202+0000","last_fullsized":"2024-04-22T07:04:14.961202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.872946+0000","last_change":"2024-04-22T07:04:12.790758+0000","last_active":"2024-04-22T07:04:39.872946+0000","last_peered":"2024-04-22T07:04:39.872946+0000","last_clean":"2024-04-22T07:04:39.872946+0000","last_became_active":"2024-04-22T07:04:12.790444+0000","last_became_peered":"2024-04-22T07:04:12.790444+0000","last_unstale":"2024-04-22T07:04:39.872946+0000","last_undegraded":"2024-04-22T07:04:39.872946+0000","last_fullsized":"2024-04-22T07:04:39.872946+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782297+0000","last_change":"2024-04-22T07:04:13.805415+0000","last_active":"2024-04-22T07:04:14.782297+0000","last_peered":"2024-04-22T07:04:14.782297+0000","last_clean":"2024-04-22T07:04:14.782297+0000","last_became_active":"2024-04-22T07:04:13.805115+0000","last_became_peered":"2024-04-22T07:04:13.805115+0000","last_unstale":"2024-04-22T07:04:14.782297+0000","last_undegraded":"2024-04-22T07:04:14.782297+0000","last_fullsized":"2024-04-22T07:04:14.782297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'17","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.875047+0000","last_change":"2024-04-22T07:04:12.789877+0000","last_active":"2024-04-22T07:04:39.875047+0000","last_peered":"2024-04-22T07:04:39.875047+0000","last_clean":"2024-04-22T07:04:39.875047+0000","last_became_active":"2024-04-22T07:04:12.789546+0000","last_became_peered":"2024-04-22T07:04:12.789546+0000","last_unstale":"2024-04-22T07:04:39.875047+0000","last_undegraded":"2024-04-22T07:04:39.875047+0000","last_fullsized":"2024-04-22T07:04:39.875047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:04.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961082+0000","last_change":"2024-04-22T07:04:13.801803+0000","last_active":"2024-04-22T07:04:14.961082+0000","last_peered":"2024-04-22T07:04:14.961082+0000","last_clean":"2024-04-22T07:04:14.961082+0000","last_became_active":"2024-04-22T07:04:13.801362+0000","last_became_peered":"2024-04-22T07:04:13.801362+0000","last_unstale":"2024-04-22T07:04:14.961082+0000","last_undegraded":"2024-04-22T07:04:14.961082+0000","last_fullsized":"2024-04-22T07:04:14.961082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.802173+0000","last_change":"2024-04-22T07:04:12.787917+0000","last_active":"2024-04-22T07:04:18.802173+0000","last_peered":"2024-04-22T07:04:18.802173+0000","last_clean":"2024-04-22T07:04:18.802173+0000","last_became_active":"2024-04-22T07:04:12.787636+0000","last_became_peered":"2024-04-22T07:04:12.787636+0000","last_unstale":"2024-04-22T07:04:18.802173+0000","last_undegraded":"2024-04-22T07:04:18.802173+0000","last_fullsized":"2024-04-22T07:04:18.802173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961456+0000","last_change":"2024-04-22T07:04:13.802786+0000","last_active":"2024-04-22T07:04:14.961456+0000","last_peered":"2024-04-22T07:04:14.961456+0000","last_clean":"2024-04-22T07:04:14.961456+0000","last_became_active":"2024-04-22T07:04:13.801299+0000","last_became_peered":"2024-04-22T07:04:13.801299+0000","last_unstale":"2024-04-22T07:04:14.961456+0000","last_undegraded":"2024-04-22T07:04:14.961456+0000","last_fullsized":"2024-04-22T07:04:14.961456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:28.362520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944029+0000","last_change":"2024-04-22T07:04:12.788593+0000","last_active":"2024-04-22T07:04:44.944029+0000","last_peered":"2024-04-22T07:04:44.944029+0000","last_clean":"2024-04-22T07:04:44.944029+0000","last_became_active":"2024-04-22T07:04:12.788314+0000","last_became_peered":"2024-04-22T07:04:12.788314+0000","last_unstale":"2024-04-22T07:04:44.944029+0000","last_undegraded":"2024-04-22T07:04:44.944029+0000","last_fullsized":"2024-04-22T07:04:44.944029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:11:27.081266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961608+0000","last_change":"2024-04-22T07:04:13.800245+0000","last_active":"2024-04-22T07:04:14.961608+0000","last_peered":"2024-04-22T07:04:14.961608+0000","last_clean":"2024-04-22T07:04:14.961608+0000","last_became_active":"2024-04-22T07:04:13.799970+0000","last_became_peered":"2024-04-22T07:04:13.799970+0000","last_unstale":"2024-04-22T07:04:14.961608+0000","last_undegraded":"2024-04-22T07:04:14.961608+0000","last_fullsized":"2024-04-22T07:04:14.961608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:13.558356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.874909+0000","last_change":"2024-04-22T07:04:12.790003+0000","last_active":"2024-04-22T07:04:39.874909+0000","last_peered":"2024-04-22T07:04:39.874909+0000","last_clean":"2024-04-22T07:04:39.874909+0000","last_became_active":"2024-04-22T07:04:12.789708+0000","last_became_peered":"2024-04-22T07:04:12.789708+0000","last_unstale":"2024-04-22T07:04:39.874909+0000","last_undegraded":"2024-04-22T07:04:39.874909+0000","last_fullsized":"2024-04-22T07:04:39.874909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783345+0000","last_change":"2024-04-22T07:04:13.795082+0000","last_active":"2024-04-22T07:04:14.783345+0000","last_peered":"2024-04-22T07:04:14.783345+0000","last_clean":"2024-04-22T07:04:14.783345+0000","last_became_active":"2024-04-22T07:04:13.794793+0000","last_became_peered":"2024-04-22T07:04:13.794793+0000","last_unstale":"2024-04-22T07:04:14.783345+0000","last_undegraded":"2024-04-22T07:04:14.783345+0000","last_fullsized":"2024-04-22T07:04:14.783345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.879536+0000","last_change":"2024-04-22T07:04:12.789252+0000","last_active":"2024-04-22T07:04:39.879536+0000","last_peered":"2024-04-22T07:04:39.879536+0000","last_clean":"2024-04-22T07:04:39.879536+0000","last_became_active":"2024-04-22T07:04:12.788946+0000","last_became_peered":"2024-04-22T07:04:12.788946+0000","last_unstale":"2024-04-22T07:04:39.879536+0000","last_undegraded":"2024-04-22T07:04:39.879536+0000","last_fullsized":"2024-04-22T07:04:39.879536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:36:13.428703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961272+0000","last_change":"2024-04-22T07:04:13.801250+0000","last_active":"2024-04-22T07:04:14.961272+0000","last_peered":"2024-04-22T07:04:14.961272+0000","last_clean":"2024-04-22T07:04:14.961272+0000","last_became_active":"2024-04-22T07:04:13.799143+0000","last_became_peered":"2024-04-22T07:04:13.799143+0000","last_unstale":"2024-04-22T07:04:14.961272+0000","last_undegraded":"2024-04-22T07:04:14.961272+0000","last_fullsized":"2024-04-22T07:04:14.961272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960952+0000","last_change":"2024-04-22T07:04:03.688462+0000","last_active":"2024-04-22T07:04:14.960952+0000","last_peered":"2024-04-22T07:04:14.960952+0000","last_clean":"2024-04-22T07:04:14.960952+0000","last_became_active":"2024-04-22T07:04:03.688191+0000","last_became_peered":"2024-04-22T07:04:03.688191+0000","last_unstale":"2024-04-22T07:04:14.960952+0000","last_undegraded":"2024-04-22T07:04:14.960952+0000","last_fullsized":"2024-04-22T07:04:14.960952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.464330+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.053999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.877201+0000","last_change":"2024-04-22T07:04:12.783751+0000","last_active":"2024-04-22T07:04:39.877201+0000","last_peered":"2024-04-22T07:04:39.877201+0000","last_clean":"2024-04-22T07:04:39.877201+0000","last_became_active":"2024-04-22T07:04:12.783460+0000","last_became_peered":"2024-04-22T07:04:12.783460+0000","last_unstale":"2024-04-22T07:04:39.877201+0000","last_undegraded":"2024-04-22T07:04:39.877201+0000","last_fullsized":"2024-04-22T07:04:39.877201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054881+0000","last_change":"2024-04-22T07:04:13.791961+0000","last_active":"2024-04-22T07:04:15.054881+0000","last_peered":"2024-04-22T07:04:15.054881+0000","last_clean":"2024-04-22T07:04:15.054881+0000","last_became_active":"2024-04-22T07:04:13.790591+0000","last_became_peered":"2024-04-22T07:04:13.790591+0000","last_unstale":"2024-04-22T07:04:15.054881+0000","last_undegraded":"2024-04-22T07:04:15.054881+0000","last_fullsized":"2024-04-22T07:04:15.054881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:33.856152+0000","last_change":"2024-04-22T07:04:12.787268+0000","last_active":"2024-04-22T07:04:33.856152+0000","last_peered":"2024-04-22T07:04:33.856152+0000","last_clean":"2024-04-22T07:04:33.856152+0000","last_became_active":"2024-04-22T07:04:12.786991+0000","last_became_peered":"2024-04-22T07:04:12.786991+0000","last_unstale":"2024-04-22T07:04:33.856152+0000","last_undegraded":"2024-04-22T07:04:33.856152+0000","last_fullsized":"2024-04-22T07:04:33.856152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054726+0000","last_change":"2024-04-22T07:04:13.800243+0000","last_active":"2024-04-22T07:04:15.054726+0000","last_peered":"2024-04-22T07:04:15.054726+0000","last_clean":"2024-04-22T07:04:15.054726+0000","last_became_active":"2024-04-22T07:04:13.799917+0000","last_became_peered":"2024-04-22T07:04:13.799917+0000","last_unstale":"2024-04-22T07:04:15.054726+0000","last_undegraded":"2024-04-22T07:04:15.054726+0000","last_fullsized":"2024-04-22T07:04:15.054726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:54.958916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944501+0000","last_change":"2024-04-22T07:04:12.787893+0000","last_active":"2024-04-22T07:04:44.944501+0000","last_peered":"2024-04-22T07:04:44.944501+0000","last_clean":"2024-04-22T07:04:44.944501+0000","last_became_active":"2024-04-22T07:04:12.787370+0000","last_became_peered":"2024-04-22T07:04:12.787370+0000","last_unstale":"2024-04-22T07:04:44.944501+0000","last_undegraded":"2024-04-22T07:04:44.944501+0000","last_fullsized":"2024-04-22T07:04:44.944501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:52:03.220670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055952+0000","last_change":"2024-04-22T07:04:13.795454+0000","last_active":"2024-04-22T07:04:15.055952+0000","last_peered":"2024-04-22T07:04:15.055952+0000","last_clean":"2024-04-22T07:04:15.055952+0000","last_became_active":"2024-04-22T07:04:13.795138+0000","last_became_peered":"2024-04-22T07:04:13.795138+0000","last_unstale":"2024-04-22T07:04:15.055952+0000","last_undegraded":"2024-04-22T07:04:15.055952+0000","last_fullsized":"2024-04-22T07:04:15.055952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944953+0000","last_change":"2024-04-22T07:04:12.786891+0000","last_active":"2024-04-22T07:04:44.944953+0000","last_peered":"2024-04-22T07:04:44.944953+0000","last_clean":"2024-04-22T07:04:44.944953+0000","last_became_active":"2024-04-22T07:04:12.786557+0000","last_became_peered":"2024-04-22T07:04:12.786557+0000","last_unstale":"2024-04-22T07:04:44.944953+0000","last_undegraded":"2024-04-22T07:04:44.944953+0000","last_fullsized":"2024-04-22T07:04:44.944953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055054+0000","last_change":"2024-04-22T07:04:13.798678+0000","last_active":"2024-04-22T07:04:15.055054+0000","last_peered":"2024-04-22T07:04:15.055054+0000","last_clean":"2024-04-22T07:04:15.055054+0000","last_became_active":"2024-04-22T07:04:13.798369+0000","last_became_peered":"2024-04-22T07:04:13.798369+0000","last_unstale":"2024-04-22T07:04:15.055054+0000","last_undegraded":"2024-04-22T07:04:15.055054+0000","last_fullsized":"2024-04-22T07:04:15.055054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:37.919431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.731812+0000","last_change":"2024-04-22T07:04:12.791897+0000","last_active":"2024-04-22T07:04:15.731812+0000","last_peered":"2024-04-22T07:04:15.731812+0000","last_clean":"2024-04-22T07:04:15.731812+0000","last_became_active":"2024-04-22T07:04:12.791580+0000","last_became_peered":"2024-04-22T07:04:12.791580+0000","last_unstale":"2024-04-22T07:04:15.731812+0000","last_undegraded":"2024-04-22T07:04:15.731812+0000","last_fullsized":"2024-04-22T07:04:15.731812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782942+0000","last_change":"2024-04-22T07:04:13.803516+0000","last_active":"2024-04-22T07:04:14.782942+0000","last_peered":"2024-04-22T07:04:14.782942+0000","last_clean":"2024-04-22T07:04:14.782942+0000","last_became_active":"2024-04-22T07:04:13.803230+0000","last_became_peered":"2024-04-22T07:04:13.803230+0000","last_unstale":"2024-04-22T07:04:14.782942+0000","last_undegraded":"2024-04-22T07:04:14.782942+0000","last_fullsized":"2024-04-22T07:04:14.782942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.877324+0000","last_change":"2024-04-22T07:04:12.789499+0000","last_active":"2024-04-22T07:04:39.877324+0000","last_peered":"2024-04-22T07:04:39.877324+0000","last_clean":"2024-04-22T07:04:39.877324+0000","last_became_active":"2024-04-22T07:04:12.789229+0000","last_became_peered":"2024-04-22T07:04:12.789229+0000","last_unstale":"2024-04-22T07:04:39.877324+0000","last_undegraded":"2024-04-22T07:04:39.877324+0000","last_fullsized":"2024-04-22T07:04:39.877324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:53.955113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055856+0000","last_change":"2024-04-22T07:04:13.797969+0000","last_active":"2024-04-22T07:04:15.055856+0000","last_peered":"2024-04-22T07:04:15.055856+0000","last_clean":"2024-04-22T07:04:15.055856+0000","last_became_active":"2024-04-22T07:04:13.797662+0000","last_became_peered":"2024-04-22T07:04:13.797662+0000","last_unstale":"2024-04-22T07:04:15.055856+0000","last_undegraded":"2024-04-22T07:04:15.055856+0000","last_fullsized":"2024-04-22T07:04:15.055856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.874965+0000","last_change":"2024-04-22T07:04:12.791787+0000","last_active":"2024-04-22T07:04:39.874965+0000","last_peered":"2024-04-22T07:04:39.874965+0000","last_clean":"2024-04-22T07:04:39.874965+0000","last_became_active":"2024-04-22T07:04:12.791460+0000","last_became_peered":"2024-04-22T07:04:12.791460+0000","last_unstale":"2024-04-22T07:04:39.874965+0000","last_undegraded":"2024-04-22T07:04:39.874965+0000","last_fullsized":"2024-04-22T07:04:39.874965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055451+0000","last_change":"2024-04-22T07:04:13.797755+0000","last_active":"2024-04-22T07:04:15.055451+0000","last_peered":"2024-04-22T07:04:15.055451+0000","last_clean":"2024-04-22T07:04:15.055451+0000","last_became_active":"2024-04-22T07:04:13.797400+0000","last_became_peered":"2024-04-22T07:04:13.797400+0000","last_unstale":"2024-04-22T07:04:15.055451+0000","last_undegraded":"2024-04-22T07:04:15.055451+0000","last_fullsized":"2024-04-22T07:04:15.055451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945268+0000","last_change":"2024-04-22T07:04:12.790458+0000","last_active":"2024-04-22T07:04:44.945268+0000","last_peered":"2024-04-22T07:04:44.945268+0000","last_clean":"2024-04-22T07:04:44.945268+0000","last_became_active":"2024-04-22T07:04:12.790209+0000","last_became_peered":"2024-04-22T07:04:12.790209+0000","last_unstale":"2024-04-22T07:04:44.945268+0000","last_undegraded":"2024-04-22T07:04:44.945268+0000","last_fullsized":"2024-04-22T07:04:44.945268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056058+0000","last_change":"2024-04-22T07:04:13.796176+0000","last_active":"2024-04-22T07:04:15.056058+0000","last_peered":"2024-04-22T07:04:15.056058+0000","last_clean":"2024-04-22T07:04:15.056058+0000","last_became_active":"2024-04-22T07:04:13.795325+0000","last_became_peered":"2024-04-22T07:04:13.795325+0000","last_unstale":"2024-04-22T07:04:15.056058+0000","last_undegraded":"2024-04-22T07:04:15.056058+0000","last_fullsized":"2024-04-22T07:04:15.056058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.832746+0000","last_change":"2024-04-22T07:04:12.791157+0000","last_active":"2024-04-22T07:04:27.832746+0000","last_peered":"2024-04-22T07:04:27.832746+0000","last_clean":"2024-04-22T07:04:27.832746+0000","last_became_active":"2024-04-22T07:04:12.790879+0000","last_became_peered":"2024-04-22T07:04:12.790879+0000","last_unstale":"2024-04-22T07:04:27.832746+0000","last_undegraded":"2024-04-22T07:04:27.832746+0000","last_fullsized":"2024-04-22T07:04:27.832746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054928+0000","last_change":"2024-04-22T07:04:13.796587+0000","last_active":"2024-04-22T07:04:15.054928+0000","last_peered":"2024-04-22T07:04:15.054928+0000","last_clean":"2024-04-22T07:04:15.054928+0000","last_became_active":"2024-04-22T07:04:13.796305+0000","last_became_peered":"2024-04-22T07:04:13.796305+0000","last_unstale":"2024-04-22T07:04:15.054928+0000","last_undegraded":"2024-04-22T07:04:15.054928+0000","last_fullsized":"2024-04-22T07:04:15.054928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:29.793237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945446+0000","last_change":"2024-04-22T07:04:12.784359+0000","last_active":"2024-04-22T07:04:44.945446+0000","last_peered":"2024-04-22T07:04:44.945446+0000","last_clean":"2024-04-22T07:04:44.945446+0000","last_became_active":"2024-04-22T07:04:12.784076+0000","last_became_peered":"2024-04-22T07:04:12.784076+0000","last_unstale":"2024-04-22T07:04:44.945446+0000","last_undegraded":"2024-04-22T07:04:44.945446+0000","last_fullsized":"2024-04-22T07:04:44.945446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054875+0000","last_change":"2024-04-22T07:04:13.802658+0000","last_active":"2024-04-22T07:04:15.054875+0000","last_peered":"2024-04-22T07:04:15.054875+0000","last_clean":"2024-04-22T07:04:15.054875+0000","last_became_active":"2024-04-22T07:04:13.802059+0000","last_became_peered":"2024-04-22T07:04:13.802059+0000","last_unstale":"2024-04-22T07:04:15.054875+0000","last_undegraded":"2024-04-22T07:04:15.054875+0000","last_fullsized":"2024-04-22T07:04:15.054875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.874697+0000","last_change":"2024-04-22T07:04:12.788090+0000","last_active":"2024-04-22T07:04:39.874697+0000","last_peered":"2024-04-22T07:04:39.874697+0000","last_clean":"2024-04-22T07:04:39.874697+0000","last_became_active":"2024-04-22T07:04:12.787818+0000","last_became_peered":"2024-04-22T07:04:12.787818+0000","last_unstale":"2024-04-22T07:04:39.874697+0000","last_undegraded":"2024-04-22T07:04:39.874697+0000","last_fullsized":"2024-04-22T07:04:39.874697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055442+0000","last_change":"2024-04-22T07:04:13.802013+0000","last_active":"2024-04-22T07:04:15.055442+0000","last_peered":"2024-04-22T07:04:15.055442+0000","last_clean":"2024-04-22T07:04:15.055442+0000","last_became_active":"2024-04-22T07:04:13.801404+0000","last_became_peered":"2024-04-22T07:04:13.801404+0000","last_unstale":"2024-04-22T07:04:15.055442+0000","last_undegraded":"2024-04-22T07:04:15.055442+0000","last_fullsized":"2024-04-22T07:04:15.055442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.830917+0000","last_change":"2024-04-22T07:04:12.790981+0000","last_active":"2024-04-22T07:04:27.830917+0000","last_peered":"2024-04-22T07:04:27.830917+0000","last_clean":"2024-04-22T07:04:27.830917+0000","last_became_active":"2024-04-22T07:04:12.790592+0000","last_became_peered":"2024-04-22T07:04:12.790592+0000","last_unstale":"2024-04-22T07:04:27.830917+0000","last_undegraded":"2024-04-22T07:04:27.830917+0000","last_fullsized":"2024-04-22T07:04:27.830917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:53.115405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781878+0000","last_change":"2024-04-22T07:04:13.800619+0000","last_active":"2024-04-22T07:04:14.781878+0000","last_peered":"2024-04-22T07:04:14.781878+0000","last_clean":"2024-04-22T07:04:14.781878+0000","last_became_active":"2024-04-22T07:04:13.800279+0000","last_became_peered":"2024-04-22T07:04:13.800279+0000","last_unstale":"2024-04-22T07:04:14.781878+0000","last_undegraded":"2024-04-22T07:04:14.781878+0000","last_fullsized":"2024-04-22T07:04:14.781878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782949+0000","last_change":"2024-04-22T07:04:13.789060+0000","last_active":"2024-04-22T07:04:14.782949+0000","last_peered":"2024-04-22T07:04:14.782949+0000","last_clean":"2024-04-22T07:04:14.782949+0000","last_became_active":"2024-04-22T07:04:13.788787+0000","last_became_peered":"2024-04-22T07:04:13.788787+0000","last_unstale":"2024-04-22T07:04:14.782949+0000","last_undegraded":"2024-04-22T07:04:14.782949+0000","last_fullsized":"2024-04-22T07:04:14.782949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944673+0000","last_change":"2024-04-22T07:04:12.785077+0000","last_active":"2024-04-22T07:04:44.944673+0000","last_peered":"2024-04-22T07:04:44.944673+0000","last_clean":"2024-04-22T07:04:44.944673+0000","last_became_active":"2024-04-22T07:04:12.784804+0000","last_became_peered":"2024-04-22T07:04:12.784804+0000","last_unstale":"2024-04-22T07:04:44.944673+0000","last_undegraded":"2024-04-22T07:04:44.944673+0000","last_fullsized":"2024-04-22T07:04:44.944673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783259+0000","last_change":"2024-04-22T07:04:13.789866+0000","last_active":"2024-04-22T07:04:14.783259+0000","last_peered":"2024-04-22T07:04:14.783259+0000","last_clean":"2024-04-22T07:04:14.783259+0000","last_became_active":"2024-04-22T07:04:13.789539+0000","last_became_peered":"2024-04-22T07:04:13.789539+0000","last_unstale":"2024-04-22T07:04:14.783259+0000","last_undegraded":"2024-04-22T07:04:14.783259+0000","last_fullsized":"2024-04-22T07:04:14.783259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'12","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945287+0000","last_change":"2024-04-22T07:04:12.786162+0000","last_active":"2024-04-22T07:04:44.945287+0000","last_peered":"2024-04-22T07:04:44.945287+0000","last_clean":"2024-04-22T07:04:44.945287+0000","last_became_active":"2024-04-22T07:04:12.785310+0000","last_became_peered":"2024-04-22T07:04:12.785310+0000","last_unstale":"2024-04-22T07:04:44.945287+0000","last_undegraded":"2024-04-22T07:04:44.945287+0000","last_fullsized":"2024-04-22T07:04:44.945287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:06.776205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056634+0000","last_change":"2024-04-22T07:04:13.796267+0000","last_active":"2024-04-22T07:04:15.056634+0000","last_peered":"2024-04-22T07:04:15.056634+0000","last_clean":"2024-04-22T07:04:15.056634+0000","last_became_active":"2024-04-22T07:04:13.795220+0000","last_became_peered":"2024-04-22T07:04:13.795220+0000","last_unstale":"2024-04-22T07:04:15.056634+0000","last_undegraded":"2024-04-22T07:04:15.056634+0000","last_fullsized":"2024-04-22T07:04:15.056634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.802121+0000","last_change":"2024-04-22T07:04:12.785962+0000","last_active":"2024-04-22T07:04:18.802121+0000","last_peered":"2024-04-22T07:04:18.802121+0000","last_clean":"2024-04-22T07:04:18.802121+0000","last_became_active":"2024-04-22T07:04:12.785608+0000","last_became_peered":"2024-04-22T07:04:12.785608+0000","last_unstale":"2024-04-22T07:04:18.802121+0000","last_undegraded":"2024-04-22T07:04:18.802121+0000","last_fullsized":"2024-04-22T07:04:18.802121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055613+0000","last_change":"2024-04-22T07:04:13.801662+0000","last_active":"2024-04-22T07:04:15.055613+0000","last_peered":"2024-04-22T07:04:15.055613+0000","last_clean":"2024-04-22T07:04:15.055613+0000","last_became_active":"2024-04-22T07:04:13.798282+0000","last_became_peered":"2024-04-22T07:04:13.798282+0000","last_unstale":"2024-04-22T07:04:15.055613+0000","last_undegraded":"2024-04-22T07:04:15.055613+0000","last_fullsized":"2024-04-22T07:04:15.055613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'13","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945261+0000","last_change":"2024-04-22T07:04:12.791445+0000","last_active":"2024-04-22T07:04:44.945261+0000","last_peered":"2024-04-22T07:04:44.945261+0000","last_clean":"2024-04-22T07:04:44.945261+0000","last_became_active":"2024-04-22T07:04:12.791150+0000","last_became_peered":"2024-04-22T07:04:12.791150+0000","last_unstale":"2024-04-22T07:04:44.945261+0000","last_undegraded":"2024-04-22T07:04:44.945261+0000","last_fullsized":"2024-04-22T07:04:44.945261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8475,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":207,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":864256,"data_stored":673703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":83637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":85167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":258048,"data_stored":82565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":83671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":82767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":856064,"data_stored":672477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":83138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:56.121 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-22T07:04:56.121 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:04:56.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.260+0000 7f148e2c1640 1 Processor -- start 2024-04-22T07:04:56.258 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.260+0000 7f148e2c1640 1 -- start start 2024-04-22T07:04:56.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f148e2c1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1488155120 0x7f14880c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.260 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f148e2c1640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1488155500 con 0x7f14880caec0 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f148e2c1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1488072f90 con 0x7f148815e8b0 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f148e2c1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f148815b0d0 con 0x7f1488155120 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f14877fe640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f14880caec0 0x7f14880d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60704/0 (socket says 172.21.15.44:60704) 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f14877fe640 1 -- 172.21.15.44:0/1628743459 learned_addr learned my addr 172.21.15.44:0/1628743459 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1487fff640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1488155120 0x7f14880c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 574290663 0 0) 0x7f1488155500 con 0x7f14880caec0 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1474003520 con 0x7f14880caec0 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2188130354 0 0) 0x7f1474003520 con 0x7f14880caec0 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1488155120 msgr2=0x7f14880c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1488155120 0x7f14880c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 >> v1:172.21.15.70:6789/0 conn(0x7f148815e8b0 legacy=0x7f148815ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148815b3b0 con 0x7f14880caec0 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3718881788 0 0) 0x7f1478001670 con 0x7f14880caec0 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 >> v1:172.21.15.44:6789/0 conn(0x7f14880caec0 legacy=0x7f14880d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.261 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1488155120 0x7f14740039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 0x7f1474004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1474004d40 0x7f14740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f148815b0d0 con 0x7f1474003f60 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.261+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f147400baa0 con 0x7f1474004d40 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.262+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f147400bca0 con 0x7f1488155120 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.262+0000 7f14877fe640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 0x7f1474004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.262+0000 7f148c85e640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1474004d40 0x7f14740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.262+0000 7f1487fff640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1488155120 0x7f14740039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.262+0000 7f14877fe640 1 -- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1488155120 msgr2=0x7f14740039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.262+0000 7f14877fe640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f1488155120 0x7f14740039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.262+0000 7f14877fe640 1 -- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1474004d40 msgr2=0x7f14740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.262+0000 7f14877fe640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1474004d40 0x7f14740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.262+0000 7f14877fe640 1 -- 172.21.15.44:0/1628743459 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1474003520 con 0x7f1474003f60 2024-04-22T07:04:56.262 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f14877fe640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 0x7f1474004800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f147800c240 tx=0x7f1474005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64eb63e5a02b009a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:56.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478007cf0 con 0x7f1474003f60 2024-04-22T07:04:56.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1478030070 con 0x7f1474003f60 2024-04-22T07:04:56.263 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f1486ffd640 1 -- 172.21.15.44:0/1628743459 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478005d70 con 0x7f1474003f60 2024-04-22T07:04:56.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f148e2c1640 1 -- 172.21.15.44:0/1628743459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 msgr2=0x7f1474004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f148e2c1640 1 --2- 172.21.15.44:0/1628743459 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 0x7f1474004800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f147800c240 tx=0x7f1474005830 comp rx=0 tx=0).stop 2024-04-22T07:04:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f148c85e640 1 -- 172.21.15.44:0/1628743459 reap_dead start 2024-04-22T07:04:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f148e2c1640 1 -- 172.21.15.44:0/1628743459 shutdown_connections 2024-04-22T07:04:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f148e2c1640 1 -- 172.21.15.44:0/1628743459 >> 172.21.15.44:0/1628743459 conn(0x7f14880676b0 msgr2=0x7f1488067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f148e2c1640 1 -- 172.21.15.44:0/1628743459 shutdown_connections 2024-04-22T07:04:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.263+0000 7f148e2c1640 1 -- 172.21.15.44:0/1628743459 wait complete. 2024-04-22T07:04:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.264+0000 7f148e2c1640 1 Processor -- start 2024-04-22T07:04:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f148e2c1640 1 -- start start 2024-04-22T07:04:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f148e2c1640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 0x7f14881452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f148e2c1640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14880ce4d0 0x7f14880c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f148e2c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1488149a20 0x7f14880c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f148e2c1640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f148815bc50 con 0x7f1474003f60 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f148e2c1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f148815b970 con 0x7f1488149a20 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f148e2c1640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f148815e390 con 0x7f14880ce4d0 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f1487fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 0x7f14881452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f1487fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 0x7f14881452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36786/0 (socket says 172.21.15.44:36786) 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f1487fff640 1 -- 172.21.15.44:0/1239347087 learned_addr learned my addr 172.21.15.44:0/1239347087 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.265+0000 7f14877fe640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14880ce4d0 0x7f14880c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f148c85e640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1488149a20 0x7f14880c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f1487fff640 1 -- 172.21.15.44:0/1239347087 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14880ce4d0 msgr2=0x7f14880c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f1487fff640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14880ce4d0 0x7f14880c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f1487fff640 1 -- 172.21.15.44:0/1239347087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1488149a20 msgr2=0x7f14880c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f1487fff640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1488149a20 0x7f14880c5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f1487fff640 1 -- 172.21.15.44:0/1239347087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1488153e10 con 0x7f1474003f60 2024-04-22T07:04:56.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f1487fff640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 0x7f14881452c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f14700119a0 tx=0x7f1470011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f1484ff9640 1 -- 172.21.15.44:0/1239347087 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147000faf0 con 0x7f1474003f60 2024-04-22T07:04:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f1484ff9640 1 -- 172.21.15.44:0/1239347087 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147000fc90 con 0x7f1474003f60 2024-04-22T07:04:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f1484ff9640 1 -- 172.21.15.44:0/1239347087 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470014440 con 0x7f1474003f60 2024-04-22T07:04:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f148e2c1640 1 -- 172.21.15.44:0/1239347087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14881540f0 con 0x7f1474003f60 2024-04-22T07:04:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.266+0000 7f148e2c1640 1 -- 172.21.15.44:0/1239347087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881546b0 con 0x7f1474003f60 2024-04-22T07:04:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.267+0000 7f148e2c1640 1 -- 172.21.15.44:0/1239347087 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148804fdc0 con 0x7f1474003f60 2024-04-22T07:04:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.268+0000 7f1484ff9640 1 -- 172.21.15.44:0/1239347087 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1470012040 con 0x7f1474003f60 2024-04-22T07:04:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.268+0000 7f1484ff9640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f146806c920 0x7f146806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.268+0000 7f1484ff9640 1 -- 172.21.15.44:0/1239347087 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f147008c120 con 0x7f1474003f60 2024-04-22T07:04:56.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.268+0000 7f14877fe640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f146806c920 0x7f146806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.268+0000 7f14877fe640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f146806c920 0x7f146806ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1478034000 tx=0x7f1478007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:56.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.272+0000 7f1484ff9640 1 -- 172.21.15.44:0/1239347087 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1470019030 con 0x7f1474003f60 2024-04-22T07:04:56.451 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.453+0000 7f148e2c1640 1 -- 172.21.15.44:0/1239347087 --> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f14880ca970 con 0x7f146806c920 2024-04-22T07:04:56.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.459+0000 7f1484ff9640 1 -- 172.21.15.44:0/1239347087 <== mgr.4101 v2:172.21.15.44:6832/1889438723 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251920 (secure 0 0 0) 0x7f1478082820 con 0x7f146806c920 2024-04-22T07:04:56.460 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:56.461 INFO:teuthology.orchestra.run.smithi044.stderr:dumped all 2024-04-22T07:04:56.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 -- 172.21.15.44:0/1239347087 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f146806c920 msgr2=0x7f146806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f146806c920 0x7f146806ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1478034000 tx=0x7f1478007000 comp rx=0 tx=0).stop 2024-04-22T07:04:56.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 -- 172.21.15.44:0/1239347087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 msgr2=0x7f14881452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 0x7f14881452c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f14700119a0 tx=0x7f1470011e60 comp rx=0 tx=0).stop 2024-04-22T07:04:56.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 -- 172.21.15.44:0/1239347087 shutdown_connections 2024-04-22T07:04:56.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f146806c920 0x7f146806ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14880ce4d0 0x7f14880c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1488149a20 0x7f14880c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 --2- 172.21.15.44:0/1239347087 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f1474003f60 0x7f14881452c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 -- 172.21.15.44:0/1239347087 >> 172.21.15.44:0/1239347087 conn(0x7f14880676b0 msgr2=0x7f14881517c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:56.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 -- 172.21.15.44:0/1239347087 shutdown_connections 2024-04-22T07:04:56.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.462+0000 7f148e2c1640 1 -- 172.21.15.44:0/1239347087 wait complete. 2024-04-22T07:04:56.471 INFO:teuthology.orchestra.run.smithi044.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-04-22T07:04:55.721228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598843,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":366,"num_write_kb":2480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240168,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734552,"statfs":{"total":773094113280,"available":772848181248,"internally_reserved":0,"allocated":3317760,"data_stored":1847125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":463,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001790"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961498+0000","last_change":"2024-04-22T07:04:13.800864+0000","last_active":"2024-04-22T07:04:14.961498+0000","last_peered":"2024-04-22T07:04:14.961498+0000","last_clean":"2024-04-22T07:04:14.961498+0000","last_became_active":"2024-04-22T07:04:13.800611+0000","last_became_peered":"2024-04-22T07:04:13.800611+0000","last_unstale":"2024-04-22T07:04:14.961498+0000","last_undegraded":"2024-04-22T07:04:14.961498+0000","last_fullsized":"2024-04-22T07:04:14.961498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:55.129127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056077+0000","last_change":"2024-04-22T07:04:13.798390+0000","last_active":"2024-04-22T07:04:15.056077+0000","last_peered":"2024-04-22T07:04:15.056077+0000","last_clean":"2024-04-22T07:04:15.056077+0000","last_became_active":"2024-04-22T07:04:13.797927+0000","last_became_peered":"2024-04-22T07:04:13.797927+0000","last_unstale":"2024-04-22T07:04:15.056077+0000","last_undegraded":"2024-04-22T07:04:15.056077+0000","last_fullsized":"2024-04-22T07:04:15.056077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056425+0000","last_change":"2024-04-22T07:04:13.795288+0000","last_active":"2024-04-22T07:04:15.056425+0000","last_peered":"2024-04-22T07:04:15.056425+0000","last_clean":"2024-04-22T07:04:15.056425+0000","last_became_active":"2024-04-22T07:04:13.794704+0000","last_became_peered":"2024-04-22T07:04:13.794704+0000","last_unstale":"2024-04-22T07:04:15.056425+0000","last_undegraded":"2024-04-22T07:04:15.056425+0000","last_fullsized":"2024-04-22T07:04:15.056425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783223+0000","last_change":"2024-04-22T07:04:13.801884+0000","last_active":"2024-04-22T07:04:14.783223+0000","last_peered":"2024-04-22T07:04:14.783223+0000","last_clean":"2024-04-22T07:04:14.783223+0000","last_became_active":"2024-04-22T07:04:13.801620+0000","last_became_peered":"2024-04-22T07:04:13.801620+0000","last_unstale":"2024-04-22T07:04:14.783223+0000","last_undegraded":"2024-04-22T07:04:14.783223+0000","last_fullsized":"2024-04-22T07:04:14.783223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961151+0000","last_change":"2024-04-22T07:04:13.798670+0000","last_active":"2024-04-22T07:04:14.961151+0000","last_peered":"2024-04-22T07:04:14.961151+0000","last_clean":"2024-04-22T07:04:14.961151+0000","last_became_active":"2024-04-22T07:04:13.798384+0000","last_became_peered":"2024-04-22T07:04:13.798384+0000","last_unstale":"2024-04-22T07:04:14.961151+0000","last_undegraded":"2024-04-22T07:04:14.961151+0000","last_fullsized":"2024-04-22T07:04:14.961151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:42:03.110079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783124+0000","last_change":"2024-04-22T07:04:13.796468+0000","last_active":"2024-04-22T07:04:14.783124+0000","last_peered":"2024-04-22T07:04:14.783124+0000","last_clean":"2024-04-22T07:04:14.783124+0000","last_became_active":"2024-04-22T07:04:13.796139+0000","last_became_peered":"2024-04-22T07:04:13.796139+0000","last_unstale":"2024-04-22T07:04:14.783124+0000","last_undegraded":"2024-04-22T07:04:14.783124+0000","last_fullsized":"2024-04-22T07:04:14.783124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961066+0000","last_change":"2024-04-22T07:04:13.801449+0000","last_active":"2024-04-22T07:04:14.961066+0000","last_peered":"2024-04-22T07:04:14.961066+0000","last_clean":"2024-04-22T07:04:14.961066+0000","last_became_active":"2024-04-22T07:04:13.799374+0000","last_became_peered":"2024-04-22T07:04:13.799374+0000","last_unstale":"2024-04-22T07:04:14.961066+0000","last_undegraded":"2024-04-22T07:04:14.961066+0000","last_fullsized":"2024-04-22T07:04:14.961066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054559+0000","last_change":"2024-04-22T07:04:13.802780+0000","last_active":"2024-04-22T07:04:15.054559+0000","last_peered":"2024-04-22T07:04:15.054559+0000","last_clean":"2024-04-22T07:04:15.054559+0000","last_became_active":"2024-04-22T07:04:13.802371+0000","last_became_peered":"2024-04-22T07:04:13.802371+0000","last_unstale":"2024-04-22T07:04:15.054559+0000","last_undegraded":"2024-04-22T07:04:15.054559+0000","last_fullsized":"2024-04-22T07:04:15.054559+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961532+0000","last_change":"2024-04-22T07:04:13.792483+0000","last_active":"2024-04-22T07:04:14.961532+0000","last_peered":"2024-04-22T07:04:14.961532+0000","last_clean":"2024-04-22T07:04:14.961532+0000","last_became_active":"2024-04-22T07:04:13.792198+0000","last_became_peered":"2024-04-22T07:04:13.792198+0000","last_unstale":"2024-04-22T07:04:14.961532+0000","last_undegraded":"2024-04-22T07:04:14.961532+0000","last_fullsized":"2024-04-22T07:04:14.961532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:04:36.943428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782040+0000","last_change":"2024-04-22T07:04:13.797379+0000","last_active":"2024-04-22T07:04:14.782040+0000","last_peered":"2024-04-22T07:04:14.782040+0000","last_clean":"2024-04-22T07:04:14.782040+0000","last_became_active":"2024-04-22T07:04:13.797029+0000","last_became_peered":"2024-04-22T07:04:13.797029+0000","last_unstale":"2024-04-22T07:04:14.782040+0000","last_undegraded":"2024-04-22T07:04:14.782040+0000","last_fullsized":"2024-04-22T07:04:14.782040+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781680+0000","last_change":"2024-04-22T07:04:13.789875+0000","last_active":"2024-04-22T07:04:14.781680+0000","last_peered":"2024-04-22T07:04:14.781680+0000","last_clean":"2024-04-22T07:04:14.781680+0000","last_became_active":"2024-04-22T07:04:13.789598+0000","last_became_peered":"2024-04-22T07:04:13.789598+0000","last_unstale":"2024-04-22T07:04:14.781680+0000","last_undegraded":"2024-04-22T07:04:14.781680+0000","last_fullsized":"2024-04-22T07:04:14.781680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055156+0000","last_change":"2024-04-22T07:04:13.791103+0000","last_active":"2024-04-22T07:04:15.055156+0000","last_peered":"2024-04-22T07:04:15.055156+0000","last_clean":"2024-04-22T07:04:15.055156+0000","last_became_active":"2024-04-22T07:04:13.790819+0000","last_became_peered":"2024-04-22T07:04:13.790819+0000","last_unstale":"2024-04-22T07:04:15.055156+0000","last_undegraded":"2024-04-22T07:04:15.055156+0000","last_fullsized":"2024-04-22T07:04:15.055156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:50.632020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055402+0000","last_change":"2024-04-22T07:04:13.802407+0000","last_active":"2024-04-22T07:04:15.055402+0000","last_peered":"2024-04-22T07:04:15.055402+0000","last_clean":"2024-04-22T07:04:15.055402+0000","last_became_active":"2024-04-22T07:04:13.801785+0000","last_became_peered":"2024-04-22T07:04:13.801785+0000","last_unstale":"2024-04-22T07:04:15.055402+0000","last_undegraded":"2024-04-22T07:04:15.055402+0000","last_fullsized":"2024-04-22T07:04:15.055402+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055557+0000","last_change":"2024-04-22T07:04:13.795888+0000","last_active":"2024-04-22T07:04:15.055557+0000","last_peered":"2024-04-22T07:04:15.055557+0000","last_clean":"2024-04-22T07:04:15.055557+0000","last_became_active":"2024-04-22T07:04:13.795092+0000","last_became_peered":"2024-04-22T07:04:13.795092+0000","last_unstale":"2024-04-22T07:04:15.055557+0000","last_undegraded":"2024-04-22T07:04:15.055557+0000","last_fullsized":"2024-04-22T07:04:15.055557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:54.119208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055926+0000","last_change":"2024-04-22T07:04:13.797138+0000","last_active":"2024-04-22T07:04:15.055926+0000","last_peered":"2024-04-22T07:04:15.055926+0000","last_clean":"2024-04-22T07:04:15.055926+0000","last_became_active":"2024-04-22T07:04:13.796763+0000","last_became_peered":"2024-04-22T07:04:13.796763+0000","last_unstale":"2024-04-22T07:04:15.055926+0000","last_undegraded":"2024-04-22T07:04:15.055926+0000","last_fullsized":"2024-04-22T07:04:15.055926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782465+0000","last_change":"2024-04-22T07:04:13.803869+0000","last_active":"2024-04-22T07:04:14.782465+0000","last_peered":"2024-04-22T07:04:14.782465+0000","last_clean":"2024-04-22T07:04:14.782465+0000","last_became_active":"2024-04-22T07:04:13.803451+0000","last_became_peered":"2024-04-22T07:04:13.803451+0000","last_unstale":"2024-04-22T07:04:14.782465+0000","last_undegraded":"2024-04-22T07:04:14.782465+0000","last_fullsized":"2024-04-22T07:04:14.782465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783077+0000","last_change":"2024-04-22T07:04:13.795746+0000","last_active":"2024-04-22T07:04:14.783077+0000","last_peered":"2024-04-22T07:04:14.783077+0000","last_clean":"2024-04-22T07:04:14.783077+0000","last_became_active":"2024-04-22T07:04:13.794694+0000","last_became_peered":"2024-04-22T07:04:13.794694+0000","last_unstale":"2024-04-22T07:04:14.783077+0000","last_undegraded":"2024-04-22T07:04:14.783077+0000","last_fullsized":"2024-04-22T07:04:14.783077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054829+0000","last_change":"2024-04-22T07:04:13.790670+0000","last_active":"2024-04-22T07:04:15.054829+0000","last_peered":"2024-04-22T07:04:15.054829+0000","last_clean":"2024-04-22T07:04:15.054829+0000","last_became_active":"2024-04-22T07:04:13.790104+0000","last_became_peered":"2024-04-22T07:04:13.790104+0000","last_unstale":"2024-04-22T07:04:15.054829+0000","last_undegraded":"2024-04-22T07:04:15.054829+0000","last_fullsized":"2024-04-22T07:04:15.054829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:09.325971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961464+0000","last_change":"2024-04-22T07:04:13.799016+0000","last_active":"2024-04-22T07:04:14.961464+0000","last_peered":"2024-04-22T07:04:14.961464+0000","last_clean":"2024-04-22T07:04:14.961464+0000","last_became_active":"2024-04-22T07:04:13.798716+0000","last_became_peered":"2024-04-22T07:04:13.798716+0000","last_unstale":"2024-04-22T07:04:14.961464+0000","last_undegraded":"2024-04-22T07:04:14.961464+0000","last_fullsized":"2024-04-22T07:04:14.961464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:47:36.609382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781586+0000","last_change":"2024-04-22T07:04:13.797296+0000","last_active":"2024-04-22T07:04:14.781586+0000","last_peered":"2024-04-22T07:04:14.781586+0000","last_clean":"2024-04-22T07:04:14.781586+0000","last_became_active":"2024-04-22T07:04:13.796997+0000","last_became_peered":"2024-04-22T07:04:13.796997+0000","last_unstale":"2024-04-22T07:04:14.781586+0000","last_undegraded":"2024-04-22T07:04:14.781586+0000","last_fullsized":"2024-04-22T07:04:14.781586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055994+0000","last_change":"2024-04-22T07:04:13.799675+0000","last_active":"2024-04-22T07:04:15.055994+0000","last_peered":"2024-04-22T07:04:15.055994+0000","last_clean":"2024-04-22T07:04:15.055994+0000","last_became_active":"2024-04-22T07:04:13.799411+0000","last_became_peered":"2024-04-22T07:04:13.799411+0000","last_unstale":"2024-04-22T07:04:15.055994+0000","last_undegraded":"2024-04-22T07:04:15.055994+0000","last_fullsized":"2024-04-22T07:04:15.055994+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783058+0000","last_change":"2024-04-22T07:04:13.796128+0000","last_active":"2024-04-22T07:04:14.783058+0000","last_peered":"2024-04-22T07:04:14.783058+0000","last_clean":"2024-04-22T07:04:14.783058+0000","last_became_active":"2024-04-22T07:04:13.795330+0000","last_became_peered":"2024-04-22T07:04:13.795330+0000","last_unstale":"2024-04-22T07:04:14.783058+0000","last_undegraded":"2024-04-22T07:04:14.783058+0000","last_fullsized":"2024-04-22T07:04:14.783058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055294+0000","last_change":"2024-04-22T07:04:13.800458+0000","last_active":"2024-04-22T07:04:15.055294+0000","last_peered":"2024-04-22T07:04:15.055294+0000","last_clean":"2024-04-22T07:04:15.055294+0000","last_became_active":"2024-04-22T07:04:13.800171+0000","last_became_peered":"2024-04-22T07:04:13.800171+0000","last_unstale":"2024-04-22T07:04:15.055294+0000","last_undegraded":"2024-04-22T07:04:15.055294+0000","last_fullsized":"2024-04-22T07:04:15.055294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782518+0000","last_change":"2024-04-22T07:04:13.802472+0000","last_active":"2024-04-22T07:04:14.782518+0000","last_peered":"2024-04-22T07:04:14.782518+0000","last_clean":"2024-04-22T07:04:14.782518+0000","last_became_active":"2024-04-22T07:04:13.802201+0000","last_became_peered":"2024-04-22T07:04:13.802201+0000","last_unstale":"2024-04-22T07:04:14.782518+0000","last_undegraded":"2024-04-22T07:04:14.782518+0000","last_fullsized":"2024-04-22T07:04:14.782518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055335+0000","last_change":"2024-04-22T07:04:13.795055+0000","last_active":"2024-04-22T07:04:15.055335+0000","last_peered":"2024-04-22T07:04:15.055335+0000","last_clean":"2024-04-22T07:04:15.055335+0000","last_became_active":"2024-04-22T07:04:13.794145+0000","last_became_peered":"2024-04-22T07:04:13.794145+0000","last_unstale":"2024-04-22T07:04:15.055335+0000","last_undegraded":"2024-04-22T07:04:15.055335+0000","last_fullsized":"2024-04-22T07:04:15.055335+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782104+0000","last_change":"2024-04-22T07:04:13.799772+0000","last_active":"2024-04-22T07:04:14.782104+0000","last_peered":"2024-04-22T07:04:14.782104+0000","last_clean":"2024-04-22T07:04:14.782104+0000","last_became_active":"2024-04-22T07:04:13.799504+0000","last_became_peered":"2024-04-22T07:04:13.799504+0000","last_unstale":"2024-04-22T07:04:14.782104+0000","last_undegraded":"2024-04-22T07:04:14.782104+0000","last_fullsized":"2024-04-22T07:04:14.782104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056133+0000","last_change":"2024-04-22T07:04:13.804604+0000","last_active":"2024-04-22T07:04:15.056133+0000","last_peered":"2024-04-22T07:04:15.056133+0000","last_clean":"2024-04-22T07:04:15.056133+0000","last_became_active":"2024-04-22T07:04:13.803594+0000","last_became_peered":"2024-04-22T07:04:13.803594+0000","last_unstale":"2024-04-22T07:04:15.056133+0000","last_undegraded":"2024-04-22T07:04:15.056133+0000","last_fullsized":"2024-04-22T07:04:15.056133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055262+0000","last_change":"2024-04-22T07:04:13.795947+0000","last_active":"2024-04-22T07:04:15.055262+0000","last_peered":"2024-04-22T07:04:15.055262+0000","last_clean":"2024-04-22T07:04:15.055262+0000","last_became_active":"2024-04-22T07:04:13.795223+0000","last_became_peered":"2024-04-22T07:04:13.795223+0000","last_unstale":"2024-04-22T07:04:15.055262+0000","last_undegraded":"2024-04-22T07:04:15.055262+0000","last_fullsized":"2024-04-22T07:04:15.055262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056195+0000","last_change":"2024-04-22T07:04:13.804225+0000","last_active":"2024-04-22T07:04:15.056195+0000","last_peered":"2024-04-22T07:04:15.056195+0000","last_clean":"2024-04-22T07:04:15.056195+0000","last_became_active":"2024-04-22T07:04:13.803920+0000","last_became_peered":"2024-04-22T07:04:13.803920+0000","last_unstale":"2024-04-22T07:04:15.056195+0000","last_undegraded":"2024-04-22T07:04:15.056195+0000","last_fullsized":"2024-04-22T07:04:15.056195+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783228+0000","last_change":"2024-04-22T07:04:13.796357+0000","last_active":"2024-04-22T07:04:14.783228+0000","last_peered":"2024-04-22T07:04:14.783228+0000","last_clean":"2024-04-22T07:04:14.783228+0000","last_became_active":"2024-04-22T07:04:13.795784+0000","last_became_peered":"2024-04-22T07:04:13.795784+0000","last_unstale":"2024-04-22T07:04:14.783228+0000","last_undegraded":"2024-04-22T07:04:14.783228+0000","last_fullsized":"2024-04-22T07:04:14.783228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054855+0000","last_change":"2024-04-22T07:04:13.784337+0000","last_active":"2024-04-22T07:04:15.054855+0000","last_peered":"2024-04-22T07:04:15.054855+0000","last_clean":"2024-04-22T07:04:15.054855+0000","last_became_active":"2024-04-22T07:04:13.783914+0000","last_became_peered":"2024-04-22T07:04:13.783914+0000","last_unstale":"2024-04-22T07:04:15.054855+0000","last_undegraded":"2024-04-22T07:04:15.054855+0000","last_fullsized":"2024-04-22T07:04:15.054855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:43:41.211395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781515+0000","last_change":"2024-04-22T07:04:13.806969+0000","last_active":"2024-04-22T07:04:14.781515+0000","last_peered":"2024-04-22T07:04:14.781515+0000","last_clean":"2024-04-22T07:04:14.781515+0000","last_became_active":"2024-04-22T07:04:13.806715+0000","last_became_peered":"2024-04-22T07:04:13.806715+0000","last_unstale":"2024-04-22T07:04:14.781515+0000","last_undegraded":"2024-04-22T07:04:14.781515+0000","last_fullsized":"2024-04-22T07:04:14.781515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:09.355833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054895+0000","last_change":"2024-04-22T07:04:13.794896+0000","last_active":"2024-04-22T07:04:15.054895+0000","last_peered":"2024-04-22T07:04:15.054895+0000","last_clean":"2024-04-22T07:04:15.054895+0000","last_became_active":"2024-04-22T07:04:13.794605+0000","last_became_peered":"2024-04-22T07:04:13.794605+0000","last_unstale":"2024-04-22T07:04:15.054895+0000","last_undegraded":"2024-04-22T07:04:15.054895+0000","last_fullsized":"2024-04-22T07:04:15.054895+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:02.613130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.836956+0000","last_change":"2024-04-22T07:04:12.786738+0000","last_active":"2024-04-22T07:04:27.836956+0000","last_peered":"2024-04-22T07:04:27.836956+0000","last_clean":"2024-04-22T07:04:27.836956+0000","last_became_active":"2024-04-22T07:04:12.786413+0000","last_became_peered":"2024-04-22T07:04:12.786413+0000","last_unstale":"2024-04-22T07:04:27.836956+0000","last_undegraded":"2024-04-22T07:04:27.836956+0000","last_fullsized":"2024-04-22T07:04:27.836956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056196+0000","last_change":"2024-04-22T07:04:13.800525+0000","last_active":"2024-04-22T07:04:15.056196+0000","last_peered":"2024-04-22T07:04:15.056196+0000","last_clean":"2024-04-22T07:04:15.056196+0000","last_became_active":"2024-04-22T07:04:13.800254+0000","last_became_peered":"2024-04-22T07:04:13.800254+0000","last_unstale":"2024-04-22T07:04:15.056196+0000","last_undegraded":"2024-04-22T07:04:15.056196+0000","last_fullsized":"2024-04-22T07:04:15.056196+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.873809+0000","last_change":"2024-04-22T07:04:12.786726+0000","last_active":"2024-04-22T07:04:39.873809+0000","last_peered":"2024-04-22T07:04:39.873809+0000","last_clean":"2024-04-22T07:04:39.873809+0000","last_became_active":"2024-04-22T07:04:12.786444+0000","last_became_peered":"2024-04-22T07:04:12.786444+0000","last_unstale":"2024-04-22T07:04:39.873809+0000","last_undegraded":"2024-04-22T07:04:39.873809+0000","last_fullsized":"2024-04-22T07:04:39.873809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:33.855518+0000","last_change":"2024-04-22T07:04:12.787357+0000","last_active":"2024-04-22T07:04:33.855518+0000","last_peered":"2024-04-22T07:04:33.855518+0000","last_clean":"2024-04-22T07:04:33.855518+0000","last_became_active":"2024-04-22T07:04:12.787066+0000","last_became_peered":"2024-04-22T07:04:12.787066+0000","last_unstale":"2024-04-22T07:04:33.855518+0000","last_undegraded":"2024-04-22T07:04:33.855518+0000","last_fullsized":"2024-04-22T07:04:33.855518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054713+0000","last_change":"2024-04-22T07:04:13.807697+0000","last_active":"2024-04-22T07:04:15.054713+0000","last_peered":"2024-04-22T07:04:15.054713+0000","last_clean":"2024-04-22T07:04:15.054713+0000","last_became_active":"2024-04-22T07:04:13.807327+0000","last_became_peered":"2024-04-22T07:04:13.807327+0000","last_unstale":"2024-04-22T07:04:15.054713+0000","last_undegraded":"2024-04-22T07:04:15.054713+0000","last_fullsized":"2024-04-22T07:04:15.054713+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:11:42.645615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.873858+0000","last_change":"2024-04-22T07:04:12.787783+0000","last_active":"2024-04-22T07:04:39.873858+0000","last_peered":"2024-04-22T07:04:39.873858+0000","last_clean":"2024-04-22T07:04:39.873858+0000","last_became_active":"2024-04-22T07:04:12.786765+0000","last_became_peered":"2024-04-22T07:04:12.786765+0000","last_unstale":"2024-04-22T07:04:39.873858+0000","last_undegraded":"2024-04-22T07:04:39.873858+0000","last_fullsized":"2024-04-22T07:04:39.873858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055740+0000","last_change":"2024-04-22T07:04:13.804931+0000","last_active":"2024-04-22T07:04:15.055740+0000","last_peered":"2024-04-22T07:04:15.055740+0000","last_clean":"2024-04-22T07:04:15.055740+0000","last_became_active":"2024-04-22T07:04:13.804629+0000","last_became_peered":"2024-04-22T07:04:13.804629+0000","last_unstale":"2024-04-22T07:04:15.055740+0000","last_undegraded":"2024-04-22T07:04:15.055740+0000","last_fullsized":"2024-04-22T07:04:15.055740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.834749+0000","last_change":"2024-04-22T07:04:12.789279+0000","last_active":"2024-04-22T07:04:27.834749+0000","last_peered":"2024-04-22T07:04:27.834749+0000","last_clean":"2024-04-22T07:04:27.834749+0000","last_became_active":"2024-04-22T07:04:12.787705+0000","last_became_peered":"2024-04-22T07:04:12.787705+0000","last_unstale":"2024-04-22T07:04:27.834749+0000","last_undegraded":"2024-04-22T07:04:27.834749+0000","last_fullsized":"2024-04-22T07:04:27.834749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782812+0000","last_change":"2024-04-22T07:04:13.794292+0000","last_active":"2024-04-22T07:04:14.782812+0000","last_peered":"2024-04-22T07:04:14.782812+0000","last_clean":"2024-04-22T07:04:14.782812+0000","last_became_active":"2024-04-22T07:04:13.793986+0000","last_became_peered":"2024-04-22T07:04:13.793986+0000","last_unstale":"2024-04-22T07:04:14.782812+0000","last_undegraded":"2024-04-22T07:04:14.782812+0000","last_fullsized":"2024-04-22T07:04:14.782812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:45.266954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944030+0000","last_change":"2024-04-22T07:04:12.787064+0000","last_active":"2024-04-22T07:04:44.944030+0000","last_peered":"2024-04-22T07:04:44.944030+0000","last_clean":"2024-04-22T07:04:44.944030+0000","last_became_active":"2024-04-22T07:04:12.786430+0000","last_became_peered":"2024-04-22T07:04:12.786430+0000","last_unstale":"2024-04-22T07:04:44.944030+0000","last_undegraded":"2024-04-22T07:04:44.944030+0000","last_fullsized":"2024-04-22T07:04:44.944030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056732+0000","last_change":"2024-04-22T07:04:13.801337+0000","last_active":"2024-04-22T07:04:15.056732+0000","last_peered":"2024-04-22T07:04:15.056732+0000","last_clean":"2024-04-22T07:04:15.056732+0000","last_became_active":"2024-04-22T07:04:13.800916+0000","last_became_peered":"2024-04-22T07:04:13.800916+0000","last_unstale":"2024-04-22T07:04:15.056732+0000","last_undegraded":"2024-04-22T07:04:15.056732+0000","last_fullsized":"2024-04-22T07:04:15.056732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.834615+0000","last_change":"2024-04-22T07:04:12.787741+0000","last_active":"2024-04-22T07:04:27.834615+0000","last_peered":"2024-04-22T07:04:27.834615+0000","last_clean":"2024-04-22T07:04:27.834615+0000","last_became_active":"2024-04-22T07:04:12.787405+0000","last_became_peered":"2024-04-22T07:04:12.787405+0000","last_unstale":"2024-04-22T07:04:27.834615+0000","last_undegraded":"2024-04-22T07:04:27.834615+0000","last_fullsized":"2024-04-22T07:04:27.834615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055184+0000","last_change":"2024-04-22T07:04:13.795502+0000","last_active":"2024-04-22T07:04:15.055184+0000","last_peered":"2024-04-22T07:04:15.055184+0000","last_clean":"2024-04-22T07:04:15.055184+0000","last_became_active":"2024-04-22T07:04:13.795199+0000","last_became_peered":"2024-04-22T07:04:13.795199+0000","last_unstale":"2024-04-22T07:04:15.055184+0000","last_undegraded":"2024-04-22T07:04:15.055184+0000","last_fullsized":"2024-04-22T07:04:15.055184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:07.780008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"23'10","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.943628+0000","last_change":"2024-04-22T07:04:12.788572+0000","last_active":"2024-04-22T07:04:44.943628+0000","last_peered":"2024-04-22T07:04:44.943628+0000","last_clean":"2024-04-22T07:04:44.943628+0000","last_became_active":"2024-04-22T07:04:12.788286+0000","last_became_peered":"2024-04-22T07:04:12.788286+0000","last_unstale":"2024-04-22T07:04:44.943628+0000","last_undegraded":"2024-04-22T07:04:44.943628+0000","last_fullsized":"2024-04-22T07:04:44.943628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961160+0000","last_change":"2024-04-22T07:04:13.804435+0000","last_active":"2024-04-22T07:04:14.961160+0000","last_peered":"2024-04-22T07:04:14.961160+0000","last_clean":"2024-04-22T07:04:14.961160+0000","last_became_active":"2024-04-22T07:04:13.804170+0000","last_became_peered":"2024-04-22T07:04:13.804170+0000","last_unstale":"2024-04-22T07:04:14.961160+0000","last_undegraded":"2024-04-22T07:04:14.961160+0000","last_fullsized":"2024-04-22T07:04:14.961160+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:27:48.782853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"23'9","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944342+0000","last_change":"2024-04-22T07:04:12.789905+0000","last_active":"2024-04-22T07:04:44.944342+0000","last_peered":"2024-04-22T07:04:44.944342+0000","last_clean":"2024-04-22T07:04:44.944342+0000","last_became_active":"2024-04-22T07:04:12.789503+0000","last_became_peered":"2024-04-22T07:04:12.789503+0000","last_unstale":"2024-04-22T07:04:44.944342+0000","last_undegraded":"2024-04-22T07:04:44.944342+0000","last_fullsized":"2024-04-22T07:04:44.944342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055741+0000","last_change":"2024-04-22T07:04:13.801490+0000","last_active":"2024-04-22T07:04:15.055741+0000","last_peered":"2024-04-22T07:04:15.055741+0000","last_clean":"2024-04-22T07:04:15.055741+0000","last_became_active":"2024-04-22T07:04:13.799356+0000","last_became_peered":"2024-04-22T07:04:13.799356+0000","last_unstale":"2024-04-22T07:04:15.055741+0000","last_undegraded":"2024-04-22T07:04:15.055741+0000","last_fullsized":"2024-04-22T07:04:15.055741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:32:04.301170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.877037+0000","last_change":"2024-04-22T07:04:12.781089+0000","last_active":"2024-04-22T07:04:39.877037+0000","last_peered":"2024-04-22T07:04:39.877037+0000","last_clean":"2024-04-22T07:04:39.877037+0000","last_became_active":"2024-04-22T07:04:12.780827+0000","last_became_peered":"2024-04-22T07:04:12.780827+0000","last_unstale":"2024-04-22T07:04:39.877037+0000","last_undegraded":"2024-04-22T07:04:39.877037+0000","last_fullsized":"2024-04-22T07:04:39.877037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:11:50.825825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961202+0000","last_change":"2024-04-22T07:04:13.801375+0000","last_active":"2024-04-22T07:04:14.961202+0000","last_peered":"2024-04-22T07:04:14.961202+0000","last_clean":"2024-04-22T07:04:14.961202+0000","last_became_active":"2024-04-22T07:04:13.799488+0000","last_became_peered":"2024-04-22T07:04:13.799488+0000","last_unstale":"2024-04-22T07:04:14.961202+0000","last_undegraded":"2024-04-22T07:04:14.961202+0000","last_fullsized":"2024-04-22T07:04:14.961202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.872946+0000","last_change":"2024-04-22T07:04:12.790758+0000","last_active":"2024-04-22T07:04:39.872946+0000","last_peered":"2024-04-22T07:04:39.872946+0000","last_clean":"2024-04-22T07:04:39.872946+0000","last_became_active":"2024-04-22T07:04:12.790444+0000","last_became_peered":"2024-04-22T07:04:12.790444+0000","last_unstale":"2024-04-22T07:04:39.872946+0000","last_undegraded":"2024-04-22T07:04:39.872946+0000","last_fullsized":"2024-04-22T07:04:39.872946+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782297+0000","last_change":"2024-04-22T07:04:13.805415+0000","last_active":"2024-04-22T07:04:14.782297+0000","last_peered":"2024-04-22T07:04:14.782297+0000","last_clean":"2024-04-22T07:04:14.782297+0000","last_became_active":"2024-04-22T07:04:13.805115+0000","last_became_peered":"2024-04-22T07:04:13.805115+0000","last_unstale":"2024-04-22T07:04:14.782297+0000","last_undegraded":"2024-04-22T07:04:14.782297+0000","last_fullsized":"2024-04-22T07:04:14.782297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"23'17","reported_seq":31,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.875047+0000","last_change":"2024-04-22T07:04:12.789877+0000","last_active":"2024-04-22T07:04:39.875047+0000","last_peered":"2024-04-22T07:04:39.875047+0000","last_clean":"2024-04-22T07:04:39.875047+0000","last_became_active":"2024-04-22T07:04:12.789546+0000","last_became_peered":"2024-04-22T07:04:12.789546+0000","last_unstale":"2024-04-22T07:04:39.875047+0000","last_undegraded":"2024-04-22T07:04:39.875047+0000","last_fullsized":"2024-04-22T07:04:39.875047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:32:04.573452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961082+0000","last_change":"2024-04-22T07:04:13.801803+0000","last_active":"2024-04-22T07:04:14.961082+0000","last_peered":"2024-04-22T07:04:14.961082+0000","last_clean":"2024-04-22T07:04:14.961082+0000","last_became_active":"2024-04-22T07:04:13.801362+0000","last_became_peered":"2024-04-22T07:04:13.801362+0000","last_unstale":"2024-04-22T07:04:14.961082+0000","last_undegraded":"2024-04-22T07:04:14.961082+0000","last_fullsized":"2024-04-22T07:04:14.961082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:26.935028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.802173+0000","last_change":"2024-04-22T07:04:12.787917+0000","last_active":"2024-04-22T07:04:18.802173+0000","last_peered":"2024-04-22T07:04:18.802173+0000","last_clean":"2024-04-22T07:04:18.802173+0000","last_became_active":"2024-04-22T07:04:12.787636+0000","last_became_peered":"2024-04-22T07:04:12.787636+0000","last_unstale":"2024-04-22T07:04:18.802173+0000","last_undegraded":"2024-04-22T07:04:18.802173+0000","last_fullsized":"2024-04-22T07:04:18.802173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:53:24.718718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961456+0000","last_change":"2024-04-22T07:04:13.802786+0000","last_active":"2024-04-22T07:04:14.961456+0000","last_peered":"2024-04-22T07:04:14.961456+0000","last_clean":"2024-04-22T07:04:14.961456+0000","last_became_active":"2024-04-22T07:04:13.801299+0000","last_became_peered":"2024-04-22T07:04:13.801299+0000","last_unstale":"2024-04-22T07:04:14.961456+0000","last_undegraded":"2024-04-22T07:04:14.961456+0000","last_fullsized":"2024-04-22T07:04:14.961456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:44:28.362520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944029+0000","last_change":"2024-04-22T07:04:12.788593+0000","last_active":"2024-04-22T07:04:44.944029+0000","last_peered":"2024-04-22T07:04:44.944029+0000","last_clean":"2024-04-22T07:04:44.944029+0000","last_became_active":"2024-04-22T07:04:12.788314+0000","last_became_peered":"2024-04-22T07:04:12.788314+0000","last_unstale":"2024-04-22T07:04:44.944029+0000","last_undegraded":"2024-04-22T07:04:44.944029+0000","last_fullsized":"2024-04-22T07:04:44.944029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:11:27.081266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961608+0000","last_change":"2024-04-22T07:04:13.800245+0000","last_active":"2024-04-22T07:04:14.961608+0000","last_peered":"2024-04-22T07:04:14.961608+0000","last_clean":"2024-04-22T07:04:14.961608+0000","last_became_active":"2024-04-22T07:04:13.799970+0000","last_became_peered":"2024-04-22T07:04:13.799970+0000","last_unstale":"2024-04-22T07:04:14.961608+0000","last_undegraded":"2024-04-22T07:04:14.961608+0000","last_fullsized":"2024-04-22T07:04:14.961608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:12:13.558356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.874909+0000","last_change":"2024-04-22T07:04:12.790003+0000","last_active":"2024-04-22T07:04:39.874909+0000","last_peered":"2024-04-22T07:04:39.874909+0000","last_clean":"2024-04-22T07:04:39.874909+0000","last_became_active":"2024-04-22T07:04:12.789708+0000","last_became_peered":"2024-04-22T07:04:12.789708+0000","last_unstale":"2024-04-22T07:04:39.874909+0000","last_undegraded":"2024-04-22T07:04:39.874909+0000","last_fullsized":"2024-04-22T07:04:39.874909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783345+0000","last_change":"2024-04-22T07:04:13.795082+0000","last_active":"2024-04-22T07:04:14.783345+0000","last_peered":"2024-04-22T07:04:14.783345+0000","last_clean":"2024-04-22T07:04:14.783345+0000","last_became_active":"2024-04-22T07:04:13.794793+0000","last_became_peered":"2024-04-22T07:04:13.794793+0000","last_unstale":"2024-04-22T07:04:14.783345+0000","last_undegraded":"2024-04-22T07:04:14.783345+0000","last_fullsized":"2024-04-22T07:04:14.783345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.879536+0000","last_change":"2024-04-22T07:04:12.789252+0000","last_active":"2024-04-22T07:04:39.879536+0000","last_peered":"2024-04-22T07:04:39.879536+0000","last_clean":"2024-04-22T07:04:39.879536+0000","last_became_active":"2024-04-22T07:04:12.788946+0000","last_became_peered":"2024-04-22T07:04:12.788946+0000","last_unstale":"2024-04-22T07:04:39.879536+0000","last_undegraded":"2024-04-22T07:04:39.879536+0000","last_fullsized":"2024-04-22T07:04:39.879536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T10:36:13.428703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1611,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.961272+0000","last_change":"2024-04-22T07:04:13.801250+0000","last_active":"2024-04-22T07:04:14.961272+0000","last_peered":"2024-04-22T07:04:14.961272+0000","last_clean":"2024-04-22T07:04:14.961272+0000","last_became_active":"2024-04-22T07:04:13.799143+0000","last_became_peered":"2024-04-22T07:04:13.799143+0000","last_unstale":"2024-04-22T07:04:14.961272+0000","last_undegraded":"2024-04-22T07:04:14.961272+0000","last_fullsized":"2024-04-22T07:04:14.961272+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:31:16.066060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.960952+0000","last_change":"2024-04-22T07:04:03.688462+0000","last_active":"2024-04-22T07:04:14.960952+0000","last_peered":"2024-04-22T07:04:14.960952+0000","last_clean":"2024-04-22T07:04:14.960952+0000","last_became_active":"2024-04-22T07:04:03.688191+0000","last_became_peered":"2024-04-22T07:04:03.688191+0000","last_unstale":"2024-04-22T07:04:14.960952+0000","last_undegraded":"2024-04-22T07:04:14.960952+0000","last_fullsized":"2024-04-22T07:04:14.960952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:02.464330+0000","last_clean_scrub_stamp":"2024-04-22T07:04:02.464330+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:10:33.053999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.877201+0000","last_change":"2024-04-22T07:04:12.783751+0000","last_active":"2024-04-22T07:04:39.877201+0000","last_peered":"2024-04-22T07:04:39.877201+0000","last_clean":"2024-04-22T07:04:39.877201+0000","last_became_active":"2024-04-22T07:04:12.783460+0000","last_became_peered":"2024-04-22T07:04:12.783460+0000","last_unstale":"2024-04-22T07:04:39.877201+0000","last_undegraded":"2024-04-22T07:04:39.877201+0000","last_fullsized":"2024-04-22T07:04:39.877201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054881+0000","last_change":"2024-04-22T07:04:13.791961+0000","last_active":"2024-04-22T07:04:15.054881+0000","last_peered":"2024-04-22T07:04:15.054881+0000","last_clean":"2024-04-22T07:04:15.054881+0000","last_became_active":"2024-04-22T07:04:13.790591+0000","last_became_peered":"2024-04-22T07:04:13.790591+0000","last_unstale":"2024-04-22T07:04:15.054881+0000","last_undegraded":"2024-04-22T07:04:15.054881+0000","last_fullsized":"2024-04-22T07:04:15.054881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:33.856152+0000","last_change":"2024-04-22T07:04:12.787268+0000","last_active":"2024-04-22T07:04:33.856152+0000","last_peered":"2024-04-22T07:04:33.856152+0000","last_clean":"2024-04-22T07:04:33.856152+0000","last_became_active":"2024-04-22T07:04:12.786991+0000","last_became_peered":"2024-04-22T07:04:12.786991+0000","last_unstale":"2024-04-22T07:04:33.856152+0000","last_undegraded":"2024-04-22T07:04:33.856152+0000","last_fullsized":"2024-04-22T07:04:33.856152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:49.628218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054726+0000","last_change":"2024-04-22T07:04:13.800243+0000","last_active":"2024-04-22T07:04:15.054726+0000","last_peered":"2024-04-22T07:04:15.054726+0000","last_clean":"2024-04-22T07:04:15.054726+0000","last_became_active":"2024-04-22T07:04:13.799917+0000","last_became_peered":"2024-04-22T07:04:13.799917+0000","last_unstale":"2024-04-22T07:04:15.054726+0000","last_undegraded":"2024-04-22T07:04:15.054726+0000","last_fullsized":"2024-04-22T07:04:15.054726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:54.958916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944501+0000","last_change":"2024-04-22T07:04:12.787893+0000","last_active":"2024-04-22T07:04:44.944501+0000","last_peered":"2024-04-22T07:04:44.944501+0000","last_clean":"2024-04-22T07:04:44.944501+0000","last_became_active":"2024-04-22T07:04:12.787370+0000","last_became_peered":"2024-04-22T07:04:12.787370+0000","last_unstale":"2024-04-22T07:04:44.944501+0000","last_undegraded":"2024-04-22T07:04:44.944501+0000","last_fullsized":"2024-04-22T07:04:44.944501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:52:03.220670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055952+0000","last_change":"2024-04-22T07:04:13.795454+0000","last_active":"2024-04-22T07:04:15.055952+0000","last_peered":"2024-04-22T07:04:15.055952+0000","last_clean":"2024-04-22T07:04:15.055952+0000","last_became_active":"2024-04-22T07:04:13.795138+0000","last_became_peered":"2024-04-22T07:04:13.795138+0000","last_unstale":"2024-04-22T07:04:15.055952+0000","last_undegraded":"2024-04-22T07:04:15.055952+0000","last_fullsized":"2024-04-22T07:04:15.055952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:38:13.704393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944953+0000","last_change":"2024-04-22T07:04:12.786891+0000","last_active":"2024-04-22T07:04:44.944953+0000","last_peered":"2024-04-22T07:04:44.944953+0000","last_clean":"2024-04-22T07:04:44.944953+0000","last_became_active":"2024-04-22T07:04:12.786557+0000","last_became_peered":"2024-04-22T07:04:12.786557+0000","last_unstale":"2024-04-22T07:04:44.944953+0000","last_undegraded":"2024-04-22T07:04:44.944953+0000","last_fullsized":"2024-04-22T07:04:44.944953+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055054+0000","last_change":"2024-04-22T07:04:13.798678+0000","last_active":"2024-04-22T07:04:15.055054+0000","last_peered":"2024-04-22T07:04:15.055054+0000","last_clean":"2024-04-22T07:04:15.055054+0000","last_became_active":"2024-04-22T07:04:13.798369+0000","last_became_peered":"2024-04-22T07:04:13.798369+0000","last_unstale":"2024-04-22T07:04:15.055054+0000","last_undegraded":"2024-04-22T07:04:15.055054+0000","last_fullsized":"2024-04-22T07:04:15.055054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:37.919431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.731812+0000","last_change":"2024-04-22T07:04:12.791897+0000","last_active":"2024-04-22T07:04:15.731812+0000","last_peered":"2024-04-22T07:04:15.731812+0000","last_clean":"2024-04-22T07:04:15.731812+0000","last_became_active":"2024-04-22T07:04:12.791580+0000","last_became_peered":"2024-04-22T07:04:12.791580+0000","last_unstale":"2024-04-22T07:04:15.731812+0000","last_undegraded":"2024-04-22T07:04:15.731812+0000","last_fullsized":"2024-04-22T07:04:15.731812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:48:01.609327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782942+0000","last_change":"2024-04-22T07:04:13.803516+0000","last_active":"2024-04-22T07:04:14.782942+0000","last_peered":"2024-04-22T07:04:14.782942+0000","last_clean":"2024-04-22T07:04:14.782942+0000","last_became_active":"2024-04-22T07:04:13.803230+0000","last_became_peered":"2024-04-22T07:04:13.803230+0000","last_unstale":"2024-04-22T07:04:14.782942+0000","last_undegraded":"2024-04-22T07:04:14.782942+0000","last_fullsized":"2024-04-22T07:04:14.782942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T09:54:35.331778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.877324+0000","last_change":"2024-04-22T07:04:12.789499+0000","last_active":"2024-04-22T07:04:39.877324+0000","last_peered":"2024-04-22T07:04:39.877324+0000","last_clean":"2024-04-22T07:04:39.877324+0000","last_became_active":"2024-04-22T07:04:12.789229+0000","last_became_peered":"2024-04-22T07:04:12.789229+0000","last_unstale":"2024-04-22T07:04:39.877324+0000","last_undegraded":"2024-04-22T07:04:39.877324+0000","last_fullsized":"2024-04-22T07:04:39.877324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:53.955113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055856+0000","last_change":"2024-04-22T07:04:13.797969+0000","last_active":"2024-04-22T07:04:15.055856+0000","last_peered":"2024-04-22T07:04:15.055856+0000","last_clean":"2024-04-22T07:04:15.055856+0000","last_became_active":"2024-04-22T07:04:13.797662+0000","last_became_peered":"2024-04-22T07:04:13.797662+0000","last_unstale":"2024-04-22T07:04:15.055856+0000","last_undegraded":"2024-04-22T07:04:15.055856+0000","last_fullsized":"2024-04-22T07:04:15.055856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:46:02.961082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.874965+0000","last_change":"2024-04-22T07:04:12.791787+0000","last_active":"2024-04-22T07:04:39.874965+0000","last_peered":"2024-04-22T07:04:39.874965+0000","last_clean":"2024-04-22T07:04:39.874965+0000","last_became_active":"2024-04-22T07:04:12.791460+0000","last_became_peered":"2024-04-22T07:04:12.791460+0000","last_unstale":"2024-04-22T07:04:39.874965+0000","last_undegraded":"2024-04-22T07:04:39.874965+0000","last_fullsized":"2024-04-22T07:04:39.874965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055451+0000","last_change":"2024-04-22T07:04:13.797755+0000","last_active":"2024-04-22T07:04:15.055451+0000","last_peered":"2024-04-22T07:04:15.055451+0000","last_clean":"2024-04-22T07:04:15.055451+0000","last_became_active":"2024-04-22T07:04:13.797400+0000","last_became_peered":"2024-04-22T07:04:13.797400+0000","last_unstale":"2024-04-22T07:04:15.055451+0000","last_undegraded":"2024-04-22T07:04:15.055451+0000","last_fullsized":"2024-04-22T07:04:15.055451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:18:36.423202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945268+0000","last_change":"2024-04-22T07:04:12.790458+0000","last_active":"2024-04-22T07:04:44.945268+0000","last_peered":"2024-04-22T07:04:44.945268+0000","last_clean":"2024-04-22T07:04:44.945268+0000","last_became_active":"2024-04-22T07:04:12.790209+0000","last_became_peered":"2024-04-22T07:04:12.790209+0000","last_unstale":"2024-04-22T07:04:44.945268+0000","last_undegraded":"2024-04-22T07:04:44.945268+0000","last_fullsized":"2024-04-22T07:04:44.945268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:24:20.343645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056058+0000","last_change":"2024-04-22T07:04:13.796176+0000","last_active":"2024-04-22T07:04:15.056058+0000","last_peered":"2024-04-22T07:04:15.056058+0000","last_clean":"2024-04-22T07:04:15.056058+0000","last_became_active":"2024-04-22T07:04:13.795325+0000","last_became_peered":"2024-04-22T07:04:13.795325+0000","last_unstale":"2024-04-22T07:04:15.056058+0000","last_undegraded":"2024-04-22T07:04:15.056058+0000","last_fullsized":"2024-04-22T07:04:15.056058+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:32:56.314946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.832746+0000","last_change":"2024-04-22T07:04:12.791157+0000","last_active":"2024-04-22T07:04:27.832746+0000","last_peered":"2024-04-22T07:04:27.832746+0000","last_clean":"2024-04-22T07:04:27.832746+0000","last_became_active":"2024-04-22T07:04:12.790879+0000","last_became_peered":"2024-04-22T07:04:12.790879+0000","last_unstale":"2024-04-22T07:04:27.832746+0000","last_undegraded":"2024-04-22T07:04:27.832746+0000","last_fullsized":"2024-04-22T07:04:27.832746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:14:36.915629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054928+0000","last_change":"2024-04-22T07:04:13.796587+0000","last_active":"2024-04-22T07:04:15.054928+0000","last_peered":"2024-04-22T07:04:15.054928+0000","last_clean":"2024-04-22T07:04:15.054928+0000","last_became_active":"2024-04-22T07:04:13.796305+0000","last_became_peered":"2024-04-22T07:04:13.796305+0000","last_unstale":"2024-04-22T07:04:15.054928+0000","last_undegraded":"2024-04-22T07:04:15.054928+0000","last_fullsized":"2024-04-22T07:04:15.054928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:29.793237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"23'7","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945446+0000","last_change":"2024-04-22T07:04:12.784359+0000","last_active":"2024-04-22T07:04:44.945446+0000","last_peered":"2024-04-22T07:04:44.945446+0000","last_clean":"2024-04-22T07:04:44.945446+0000","last_became_active":"2024-04-22T07:04:12.784076+0000","last_became_peered":"2024-04-22T07:04:12.784076+0000","last_unstale":"2024-04-22T07:04:44.945446+0000","last_undegraded":"2024-04-22T07:04:44.945446+0000","last_fullsized":"2024-04-22T07:04:44.945446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.054875+0000","last_change":"2024-04-22T07:04:13.802658+0000","last_active":"2024-04-22T07:04:15.054875+0000","last_peered":"2024-04-22T07:04:15.054875+0000","last_clean":"2024-04-22T07:04:15.054875+0000","last_became_active":"2024-04-22T07:04:13.802059+0000","last_became_peered":"2024-04-22T07:04:13.802059+0000","last_unstale":"2024-04-22T07:04:15.054875+0000","last_undegraded":"2024-04-22T07:04:15.054875+0000","last_fullsized":"2024-04-22T07:04:15.054875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:39.874697+0000","last_change":"2024-04-22T07:04:12.788090+0000","last_active":"2024-04-22T07:04:39.874697+0000","last_peered":"2024-04-22T07:04:39.874697+0000","last_clean":"2024-04-22T07:04:39.874697+0000","last_became_active":"2024-04-22T07:04:12.787818+0000","last_became_peered":"2024-04-22T07:04:12.787818+0000","last_unstale":"2024-04-22T07:04:39.874697+0000","last_undegraded":"2024-04-22T07:04:39.874697+0000","last_fullsized":"2024-04-22T07:04:39.874697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T11:28:48.711731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055442+0000","last_change":"2024-04-22T07:04:13.802013+0000","last_active":"2024-04-22T07:04:15.055442+0000","last_peered":"2024-04-22T07:04:15.055442+0000","last_clean":"2024-04-22T07:04:15.055442+0000","last_became_active":"2024-04-22T07:04:13.801404+0000","last_became_peered":"2024-04-22T07:04:13.801404+0000","last_unstale":"2024-04-22T07:04:15.055442+0000","last_undegraded":"2024-04-22T07:04:15.055442+0000","last_fullsized":"2024-04-22T07:04:15.055442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:27.830917+0000","last_change":"2024-04-22T07:04:12.790981+0000","last_active":"2024-04-22T07:04:27.830917+0000","last_peered":"2024-04-22T07:04:27.830917+0000","last_clean":"2024-04-22T07:04:27.830917+0000","last_became_active":"2024-04-22T07:04:12.790592+0000","last_became_peered":"2024-04-22T07:04:12.790592+0000","last_unstale":"2024-04-22T07:04:27.830917+0000","last_undegraded":"2024-04-22T07:04:27.830917+0000","last_fullsized":"2024-04-22T07:04:27.830917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:05:53.115405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.781878+0000","last_change":"2024-04-22T07:04:13.800619+0000","last_active":"2024-04-22T07:04:14.781878+0000","last_peered":"2024-04-22T07:04:14.781878+0000","last_clean":"2024-04-22T07:04:14.781878+0000","last_became_active":"2024-04-22T07:04:13.800279+0000","last_became_peered":"2024-04-22T07:04:13.800279+0000","last_unstale":"2024-04-22T07:04:14.781878+0000","last_undegraded":"2024-04-22T07:04:14.781878+0000","last_fullsized":"2024-04-22T07:04:14.781878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:36:57.331486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.782949+0000","last_change":"2024-04-22T07:04:13.789060+0000","last_active":"2024-04-22T07:04:14.782949+0000","last_peered":"2024-04-22T07:04:14.782949+0000","last_clean":"2024-04-22T07:04:14.782949+0000","last_became_active":"2024-04-22T07:04:13.788787+0000","last_became_peered":"2024-04-22T07:04:13.788787+0000","last_unstale":"2024-04-22T07:04:14.782949+0000","last_undegraded":"2024-04-22T07:04:14.782949+0000","last_fullsized":"2024-04-22T07:04:14.782949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:08:45.906489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"23'5","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.944673+0000","last_change":"2024-04-22T07:04:12.785077+0000","last_active":"2024-04-22T07:04:44.944673+0000","last_peered":"2024-04-22T07:04:44.944673+0000","last_clean":"2024-04-22T07:04:44.944673+0000","last_became_active":"2024-04-22T07:04:12.784804+0000","last_became_peered":"2024-04-22T07:04:12.784804+0000","last_unstale":"2024-04-22T07:04:44.944673+0000","last_undegraded":"2024-04-22T07:04:44.944673+0000","last_fullsized":"2024-04-22T07:04:44.944673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T07:57:28.789434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:14.783259+0000","last_change":"2024-04-22T07:04:13.789866+0000","last_active":"2024-04-22T07:04:14.783259+0000","last_peered":"2024-04-22T07:04:14.783259+0000","last_clean":"2024-04-22T07:04:14.783259+0000","last_became_active":"2024-04-22T07:04:13.789539+0000","last_became_peered":"2024-04-22T07:04:13.789539+0000","last_unstale":"2024-04-22T07:04:14.783259+0000","last_undegraded":"2024-04-22T07:04:14.783259+0000","last_fullsized":"2024-04-22T07:04:14.783259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T12:12:10.869843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"23'12","reported_seq":26,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945287+0000","last_change":"2024-04-22T07:04:12.786162+0000","last_active":"2024-04-22T07:04:44.945287+0000","last_peered":"2024-04-22T07:04:44.945287+0000","last_clean":"2024-04-22T07:04:44.945287+0000","last_became_active":"2024-04-22T07:04:12.785310+0000","last_became_peered":"2024-04-22T07:04:12.785310+0000","last_unstale":"2024-04-22T07:04:44.945287+0000","last_undegraded":"2024-04-22T07:04:44.945287+0000","last_fullsized":"2024-04-22T07:04:44.945287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:44:06.776205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.056634+0000","last_change":"2024-04-22T07:04:13.796267+0000","last_active":"2024-04-22T07:04:15.056634+0000","last_peered":"2024-04-22T07:04:15.056634+0000","last_clean":"2024-04-22T07:04:15.056634+0000","last_became_active":"2024-04-22T07:04:13.795220+0000","last_became_peered":"2024-04-22T07:04:13.795220+0000","last_unstale":"2024-04-22T07:04:15.056634+0000","last_undegraded":"2024-04-22T07:04:15.056634+0000","last_fullsized":"2024-04-22T07:04:15.056634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T08:21:03.703713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:18.802121+0000","last_change":"2024-04-22T07:04:12.785962+0000","last_active":"2024-04-22T07:04:18.802121+0000","last_peered":"2024-04-22T07:04:18.802121+0000","last_clean":"2024-04-22T07:04:18.802121+0000","last_became_active":"2024-04-22T07:04:12.785608+0000","last_became_peered":"2024-04-22T07:04:12.785608+0000","last_unstale":"2024-04-22T07:04:18.802121+0000","last_undegraded":"2024-04-22T07:04:18.802121+0000","last_fullsized":"2024-04-22T07:04:18.802121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:54:08.457822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:15.055613+0000","last_change":"2024-04-22T07:04:13.801662+0000","last_active":"2024-04-22T07:04:15.055613+0000","last_peered":"2024-04-22T07:04:15.055613+0000","last_clean":"2024-04-22T07:04:15.055613+0000","last_became_active":"2024-04-22T07:04:13.798282+0000","last_became_peered":"2024-04-22T07:04:13.798282+0000","last_unstale":"2024-04-22T07:04:15.055613+0000","last_undegraded":"2024-04-22T07:04:15.055613+0000","last_fullsized":"2024-04-22T07:04:15.055613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:12.766561+0000","last_clean_scrub_stamp":"2024-04-22T07:04:12.766561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:21:18.193184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"23'13","reported_seq":27,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-22T07:04:44.945261+0000","last_change":"2024-04-22T07:04:12.791445+0000","last_active":"2024-04-22T07:04:44.945261+0000","last_peered":"2024-04-22T07:04:44.945261+0000","last_clean":"2024-04-22T07:04:44.945261+0000","last_became_active":"2024-04-22T07:04:12.791150+0000","last_became_peered":"2024-04-22T07:04:12.791150+0000","last_unstale":"2024-04-22T07:04:44.945261+0000","last_undegraded":"2024-04-22T07:04:44.945261+0000","last_fullsized":"2024-04-22T07:04:44.945261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T07:04:11.762758+0000","last_clean_scrub_stamp":"2024-04-22T07:04:11.762758+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:20:51.342445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8475,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":207,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":864256,"data_stored":673703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":83637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":282624,"data_stored":85167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31428,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340412,"statfs":{"total":96636764160,"available":96604581888,"internally_reserved":0,"allocated":258048,"data_stored":82565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":83671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":82767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":856064,"data_stored":672477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":83138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T07:04:56.474 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.614+0000 7f5fd1e25640 1 Processor -- start 2024-04-22T07:04:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.614+0000 7f5fd1e25640 1 -- start start 2024-04-22T07:04:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fd1e25640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5fcc15ad30 0x7f5fcc15e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fd1e25640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fcc15ee90 con 0x7f5fcc0cd6f0 2024-04-22T07:04:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fd1e25640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fcc15f170 con 0x7f5fcc154f70 2024-04-22T07:04:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fd1e25640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f5fcc0cd260 con 0x7f5fcc15ad30 2024-04-22T07:04:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fcbfff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f5fcc15ad30 0x7f5fcc15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fcaffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f5fcc0cd6f0 0x7f5fcc0cb3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60716/0 (socket says 172.21.15.44:60716) 2024-04-22T07:04:56.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fcaffd640 1 -- 172.21.15.44:0/2626722622 learned_addr learned my addr 172.21.15.44:0/2626722622 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:56.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fcbfff640 1 -- 172.21.15.44:0/2626722622 >> v1:172.21.15.70:6789/0 conn(0x7f5fcc154f70 legacy=0x7f5fcc0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:56.615 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fcbfff640 1 -- 172.21.15.44:0/2626722622 >> v1:172.21.15.44:6789/0 conn(0x7f5fcc0cd6f0 legacy=0x7f5fcc0cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fcbfff640 1 -- 172.21.15.44:0/2626722622 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc0cc930 con 0x7f5fcc15ad30 2024-04-22T07:04:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.615+0000 7f5fca7fc640 1 -- 172.21.15.44:0/2626722622 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 442890005 0 0) 0x7f5fcc15ee90 con 0x7f5fcc0cd6f0 2024-04-22T07:04:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.616+0000 7f5fcbfff640 1 --2- 172.21.15.44:0/2626722622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15ad30 0x7f5fcc15e300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009d90 tx=0x7f5fc402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94fb53f5acdb4b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.616+0000 7f5fca7fc640 1 -- 172.21.15.44:0/2626722622 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4039070 con 0x7f5fcc15ad30 2024-04-22T07:04:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.616+0000 7f5fca7fc640 1 -- 172.21.15.44:0/2626722622 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc402cd00 con 0x7f5fcc15ad30 2024-04-22T07:04:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.616+0000 7f5fca7fc640 1 -- 172.21.15.44:0/2626722622 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4034660 con 0x7f5fcc15ad30 2024-04-22T07:04:56.616 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.617+0000 7f5fd1e25640 1 -- 172.21.15.44:0/2626722622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15ad30 msgr2=0x7f5fcc15e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.617+0000 7f5fd1e25640 1 --2- 172.21.15.44:0/2626722622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15ad30 0x7f5fcc15e300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009d90 tx=0x7f5fc402bf50 comp rx=0 tx=0).stop 2024-04-22T07:04:56.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.617+0000 7f5fd1e25640 1 -- 172.21.15.44:0/2626722622 shutdown_connections 2024-04-22T07:04:56.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.617+0000 7f5fd1e25640 1 --2- 172.21.15.44:0/2626722622 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc15ad30 0x7f5fcc15e300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.617+0000 7f5fd1e25640 1 -- 172.21.15.44:0/2626722622 >> 172.21.15.44:0/2626722622 conn(0x7f5fcc067d40 msgr2=0x7f5fcc06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:56.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.617+0000 7f5fd1e25640 1 -- 172.21.15.44:0/2626722622 shutdown_connections 2024-04-22T07:04:56.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.617+0000 7f5fd1e25640 1 -- 172.21.15.44:0/2626722622 wait complete. 2024-04-22T07:04:56.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.618+0000 7f5fd1e25640 1 Processor -- start 2024-04-22T07:04:56.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.618+0000 7f5fd1e25640 1 -- start start 2024-04-22T07:04:56.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fd1e25640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fcc0cd6f0 0x7f5fcc1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fd1e25640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc154f70 0x7f5fcc14b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fd1e25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc15ad30 0x7f5fcc157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fd1e25640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc155350 con 0x7f5fcc154f70 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fd1e25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc0ccef0 con 0x7f5fcc15ad30 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fd1e25640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5fcc157ad0 con 0x7f5fcc0cd6f0 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fcaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc154f70 0x7f5fcc14b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fcaffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc154f70 0x7f5fcc14b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36794/0 (socket says 172.21.15.44:36794) 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fcaffd640 1 -- 172.21.15.44:0/3917128768 learned_addr learned my addr 172.21.15.44:0/3917128768 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fcb7fe640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fcc0cd6f0 0x7f5fcc1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fcbfff640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc15ad30 0x7f5fcc157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fcaffd640 1 -- 172.21.15.44:0/3917128768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fcc0cd6f0 msgr2=0x7f5fcc1f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fcaffd640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fcc0cd6f0 0x7f5fcc1f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fcaffd640 1 -- 172.21.15.44:0/3917128768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc15ad30 msgr2=0x7f5fcc157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fcaffd640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc15ad30 0x7f5fcc157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.622 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.619+0000 7f5fcaffd640 1 -- 172.21.15.44:0/3917128768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc1f5230 con 0x7f5fcc154f70 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.620+0000 7f5fcaffd640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc154f70 0x7f5fcc14b220 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c430 tx=0x7f5fc000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.620+0000 7f5fb3fff640 1 -- 172.21.15.44:0/3917128768 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00074e0 con 0x7f5fcc154f70 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.620+0000 7f5fb3fff640 1 -- 172.21.15.44:0/3917128768 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc00047e0 con 0x7f5fcc154f70 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.620+0000 7f5fd1e25640 1 -- 172.21.15.44:0/3917128768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fcc14b760 con 0x7f5fcc154f70 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.620+0000 7f5fb3fff640 1 -- 172.21.15.44:0/3917128768 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0004bd0 con 0x7f5fcc154f70 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.620+0000 7f5fd1e25640 1 -- 172.21.15.44:0/3917128768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fcc155ac0 con 0x7f5fcc154f70 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.621+0000 7f5fd1e25640 1 -- 172.21.15.44:0/3917128768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fcc0b3280 con 0x7f5fcc154f70 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.621+0000 7f5fb3fff640 1 -- 172.21.15.44:0/3917128768 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fc0011070 con 0x7f5fcc154f70 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.622+0000 7f5fb3fff640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5fb806c8f0 0x7f5fb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.622+0000 7f5fb3fff640 1 -- 172.21.15.44:0/3917128768 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f5fc008b750 con 0x7f5fcc154f70 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.622+0000 7f5fcb7fe640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5fb806c8f0 0x7f5fb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.623 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.622+0000 7f5fcb7fe640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5fb806c8f0 0x7f5fb806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc004ea0 tx=0x7f5fbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:56.624 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.626+0000 7f5fb3fff640 1 -- 172.21.15.44:0/3917128768 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5fcc14b760 con 0x7f5fcc154f70 2024-04-22T07:04:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.848+0000 7f5fd1e25640 1 -- 172.21.15.44:0/3917128768 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fcc04fdc0 con 0x7f5fcc154f70 2024-04-22T07:04:56.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.848+0000 7f5fb3fff640 1 -- 172.21.15.44:0/3917128768 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6908 (secure 0 0 0) 0x7f5fc000e280 con 0x7f5fcc154f70 2024-04-22T07:04:56.846 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:56.847 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:04:44.948538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4324,"mds_1":4303,"mds_2":4300,"mds_3":4278,"mds_4":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"b","rank":3,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4324,"qdb_cluster":[4275,4278,4300,4303,4324]},"id":1}]} 2024-04-22T07:04:56.847 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-04-22T07:04:56.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.850+0000 7f5fd1e25640 1 -- 172.21.15.44:0/3917128768 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5fb806c8f0 msgr2=0x7f5fb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.848 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.850+0000 7f5fd1e25640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5fb806c8f0 0x7f5fb806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc004ea0 tx=0x7f5fbc009040 comp rx=0 tx=0).stop 2024-04-22T07:04:56.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.850+0000 7f5fd1e25640 1 -- 172.21.15.44:0/3917128768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc154f70 msgr2=0x7f5fcc14b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.850+0000 7f5fd1e25640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc154f70 0x7f5fcc14b220 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c430 tx=0x7f5fc000c8f0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.850+0000 7f5fd1e25640 1 -- 172.21.15.44:0/3917128768 shutdown_connections 2024-04-22T07:04:56.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.850+0000 7f5fd1e25640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f5fb806c8f0 0x7f5fb806edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.850+0000 7f5fd1e25640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc15ad30 0x7f5fcc157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.850+0000 7f5fd1e25640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5fcc154f70 0x7f5fcc14b220 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.850+0000 7f5fd1e25640 1 --2- 172.21.15.44:0/3917128768 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f5fcc0cd6f0 0x7f5fcc1f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.850+0000 7f5fd1e25640 1 -- 172.21.15.44:0/3917128768 >> 172.21.15.44:0/3917128768 conn(0x7f5fcc067d40 msgr2=0x7f5fcc159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:56.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.851+0000 7f5fd1e25640 1 -- 172.21.15.44:0/3917128768 shutdown_connections 2024-04-22T07:04:56.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.851+0000 7f5fd1e25640 1 -- 172.21.15.44:0/3917128768 wait complete. 2024-04-22T07:04:56.859 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:56.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.963+0000 7f29de9ca640 1 Processor -- start 2024-04-22T07:04:56.961 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.963+0000 7f29de9ca640 1 -- start start 2024-04-22T07:04:56.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29de9ca640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29d80caec0 0x7f29d80c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29de9ca640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d815e8b0 0x7f29d815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29de9ca640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d8155500 con 0x7f29d8155120 2024-04-22T07:04:56.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29de9ca640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f29d8072f90 con 0x7f29d80caec0 2024-04-22T07:04:56.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29de9ca640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f29d815f220 con 0x7f29d815e8b0 2024-04-22T07:04:56.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29dcf67640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d815e8b0 0x7f29d815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29d77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29d80caec0 0x7f29d80c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29d7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f29d8155120 0x7f29d80d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:53292/0 (socket says 172.21.15.44:53292) 2024-04-22T07:04:56.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29d7fff640 1 -- 172.21.15.44:0/3320557927 learned_addr learned my addr 172.21.15.44:0/3320557927 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:56.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29dcf67640 1 -- 172.21.15.44:0/3320557927 >> v2:172.21.15.70:3300/0 conn(0x7f29d80caec0 msgr2=0x7f29d80c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:56.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29dcf67640 1 --2- 172.21.15.44:0/3320557927 >> v2:172.21.15.70:3300/0 conn(0x7f29d80caec0 0x7f29d80c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29dcf67640 1 -- 172.21.15.44:0/3320557927 >> v1:172.21.15.70:6789/0 conn(0x7f29d8155120 legacy=0x7f29d80d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.964+0000 7f29dcf67640 1 -- 172.21.15.44:0/3320557927 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d815b0d0 con 0x7f29d815e8b0 2024-04-22T07:04:56.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.965+0000 7f29dcf67640 1 --2- 172.21.15.44:0/3320557927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d815e8b0 0x7f29d815ece0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009d90 tx=0x7f29d002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33acf2f2f0514b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:56.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.965+0000 7f29d6ffd640 1 -- 172.21.15.44:0/3320557927 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0039070 con 0x7f29d815e8b0 2024-04-22T07:04:56.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.965+0000 7f29d6ffd640 1 -- 172.21.15.44:0/3320557927 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d002cc50 con 0x7f29d815e8b0 2024-04-22T07:04:56.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.965+0000 7f29d6ffd640 1 -- 172.21.15.44:0/3320557927 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0034440 con 0x7f29d815e8b0 2024-04-22T07:04:56.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.966+0000 7f29de9ca640 1 -- 172.21.15.44:0/3320557927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d815e8b0 msgr2=0x7f29d815ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.966+0000 7f29de9ca640 1 --2- 172.21.15.44:0/3320557927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d815e8b0 0x7f29d815ece0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009d90 tx=0x7f29d002bea0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.966+0000 7f29de9ca640 1 -- 172.21.15.44:0/3320557927 shutdown_connections 2024-04-22T07:04:56.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.966+0000 7f29de9ca640 1 --2- 172.21.15.44:0/3320557927 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d815e8b0 0x7f29d815ece0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.966+0000 7f29de9ca640 1 --2- 172.21.15.44:0/3320557927 >> v2:172.21.15.70:3300/0 conn(0x7f29d80caec0 0x7f29d80c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.966+0000 7f29de9ca640 1 -- 172.21.15.44:0/3320557927 >> 172.21.15.44:0/3320557927 conn(0x7f29d80676b0 msgr2=0x7f29d8067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:56.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.966+0000 7f29de9ca640 1 -- 172.21.15.44:0/3320557927 shutdown_connections 2024-04-22T07:04:56.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.966+0000 7f29de9ca640 1 -- 172.21.15.44:0/3320557927 wait complete. 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.967+0000 7f29de9ca640 1 Processor -- start 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.967+0000 7f29de9ca640 1 -- start start 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29de9ca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d80caec0 0x7f29d8150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29de9ca640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d8155120 0x7f29d81f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29de9ca640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29d815e8b0 0x7f29d814e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29de9ca640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f29d815b970 con 0x7f29d815e8b0 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29de9ca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f29d815b690 con 0x7f29d80caec0 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29de9ca640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f29d815bc50 con 0x7f29d8155120 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29dcf67640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29d815e8b0 0x7f29d814e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29dcf67640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29d815e8b0 0x7f29d814e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36806/0 (socket says 172.21.15.44:36806) 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29dcf67640 1 -- 172.21.15.44:0/2739513396 learned_addr learned my addr 172.21.15.44:0/2739513396 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:56.967 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29d77fe640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d8155120 0x7f29d81f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29d7fff640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d80caec0 0x7f29d8150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29dcf67640 1 -- 172.21.15.44:0/2739513396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d8155120 msgr2=0x7f29d81f1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.968+0000 7f29dcf67640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d8155120 0x7f29d81f1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.969+0000 7f29dcf67640 1 -- 172.21.15.44:0/2739513396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d80caec0 msgr2=0x7f29d8150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.969+0000 7f29dcf67640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d80caec0 0x7f29d8150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.969+0000 7f29dcf67640 1 -- 172.21.15.44:0/2739513396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d8158b80 con 0x7f29d815e8b0 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.969+0000 7f29dcf67640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29d815e8b0 0x7f29d814e0b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009eb0 tx=0x7f29d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.969+0000 7f29d4ff9640 1 -- 172.21.15.44:0/2739513396 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0039050 con 0x7f29d815e8b0 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.969+0000 7f29d4ff9640 1 -- 172.21.15.44:0/2739513396 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d0045070 con 0x7f29d815e8b0 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.969+0000 7f29de9ca640 1 -- 172.21.15.44:0/2739513396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d8158e60 con 0x7f29d815e8b0 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.969+0000 7f29d4ff9640 1 -- 172.21.15.44:0/2739513396 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0009450 con 0x7f29d815e8b0 2024-04-22T07:04:56.968 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.969+0000 7f29de9ca640 1 -- 172.21.15.44:0/2739513396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d815aae0 con 0x7f29d815e8b0 2024-04-22T07:04:56.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.970+0000 7f29de9ca640 1 -- 172.21.15.44:0/2739513396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d8072440 con 0x7f29d815e8b0 2024-04-22T07:04:56.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.971+0000 7f29d4ff9640 1 -- 172.21.15.44:0/2739513396 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29d003e610 con 0x7f29d815e8b0 2024-04-22T07:04:56.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.971+0000 7f29d4ff9640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f29c406c8f0 0x7f29c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:56.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.971+0000 7f29d4ff9640 1 -- 172.21.15.44:0/2739513396 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f29d00ae550 con 0x7f29d815e8b0 2024-04-22T07:04:56.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.971+0000 7f29d7fff640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f29c406c8f0 0x7f29c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:56.969 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.972+0000 7f29d7fff640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f29c406c8f0 0x7f29c406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f29c8007fd0 tx=0x7f29c8007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:56.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:56.976+0000 7f29d4ff9640 1 -- 172.21.15.44:0/2739513396 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f29d8072440 con 0x7f29d815e8b0 2024-04-22T07:04:57.160 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.163+0000 7f29de9ca640 1 -- 172.21.15.44:0/2739513396 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29d804fdc0 con 0x7f29d815e8b0 2024-04-22T07:04:57.162 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.164+0000 7f29d4ff9640 1 -- 172.21.15.44:0/2739513396 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16807 (secure 0 0 0) 0x7f29d00ace20 con 0x7f29d815e8b0 2024-04-22T07:04:57.162 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:57.162 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:54.977439+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:57.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 -- 172.21.15.44:0/2739513396 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f29c406c8f0 msgr2=0x7f29c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f29c406c8f0 0x7f29c406edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f29c8007fd0 tx=0x7f29c8007e80 comp rx=0 tx=0).stop 2024-04-22T07:04:57.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 -- 172.21.15.44:0/2739513396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29d815e8b0 msgr2=0x7f29d814e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29d815e8b0 0x7f29d814e0b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f29d0009eb0 tx=0x7f29d0004a40 comp rx=0 tx=0).stop 2024-04-22T07:04:57.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 -- 172.21.15.44:0/2739513396 shutdown_connections 2024-04-22T07:04:57.164 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f29c406c8f0 0x7f29c406edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f29d8155120 0x7f29d81f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f29d815e8b0 0x7f29d814e0b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 --2- 172.21.15.44:0/2739513396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29d80caec0 0x7f29d8150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 -- 172.21.15.44:0/2739513396 >> 172.21.15.44:0/2739513396 conn(0x7f29d80676b0 msgr2=0x7f29d8062390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 -- 172.21.15.44:0/2739513396 shutdown_connections 2024-04-22T07:04:57.165 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.166+0000 7f29de9ca640 1 -- 172.21.15.44:0/2739513396 wait complete. 2024-04-22T07:04:57.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:04:01.723984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:11.652423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:12.098854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:04:57.176 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:04:57.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.281+0000 7faf1b723640 1 Processor -- start 2024-04-22T07:04:57.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.281+0000 7faf1b723640 1 -- start start 2024-04-22T07:04:57.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.282+0000 7faf1b723640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf14070b30 0x7faf1406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.282+0000 7faf1b723640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7faf1406c250 0x7faf14154960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.282+0000 7faf1b723640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf14073890 con 0x7faf14070750 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.282+0000 7faf1b723640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7faf140cb790 con 0x7faf1406c250 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.282+0000 7faf1b723640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7faf1406beb0 con 0x7faf14070b30 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.282+0000 7faf19cc0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7faf1406c250 0x7faf14154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.282+0000 7faf19cc0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7faf1406c250 0x7faf14154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36810/0 (socket says 172.21.15.44:36810) 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.282+0000 7faf19cc0640 1 -- 172.21.15.44:0/1391248818 learned_addr learned my addr 172.21.15.44:0/1391248818 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.283+0000 7faf18cbe640 1 --2- 172.21.15.44:0/1391248818 >> v2:172.21.15.70:3300/0 conn(0x7faf14070b30 0x7faf1406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.283+0000 7faf19cc0640 1 -- 172.21.15.44:0/1391248818 >> v2:172.21.15.70:3300/0 conn(0x7faf14070b30 msgr2=0x7faf1406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.283+0000 7faf19cc0640 1 --2- 172.21.15.44:0/1391248818 >> v2:172.21.15.70:3300/0 conn(0x7faf14070b30 0x7faf1406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.283+0000 7faf19cc0640 1 -- 172.21.15.44:0/1391248818 >> v1:172.21.15.70:6789/0 conn(0x7faf14070750 legacy=0x7faf140cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.283+0000 7faf19cc0640 1 -- 172.21.15.44:0/1391248818 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf140cb1f0 con 0x7faf1406c250 2024-04-22T07:04:57.281 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.283+0000 7faf18cbe640 1 --2- 172.21.15.44:0/1391248818 >> v2:172.21.15.70:3300/0 conn(0x7faf14070b30 0x7faf1406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.283+0000 7faf19cc0640 1 --2- 172.21.15.44:0/1391248818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf1406c250 0x7faf14154960 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf140cad80 tx=0x7faf0402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0570dc5654cbde server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf13fff640 1 -- 172.21.15.44:0/1391248818 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0402cbb0 con 0x7faf1406c250 2024-04-22T07:04:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf13fff640 1 -- 172.21.15.44:0/1391248818 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0402cd50 con 0x7faf1406c250 2024-04-22T07:04:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf13fff640 1 -- 172.21.15.44:0/1391248818 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04034610 con 0x7faf1406c250 2024-04-22T07:04:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf1b723640 1 -- 172.21.15.44:0/1391248818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf1406c250 msgr2=0x7faf14154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf1b723640 1 --2- 172.21.15.44:0/1391248818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf1406c250 0x7faf14154960 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7faf140cad80 tx=0x7faf0402c010 comp rx=0 tx=0).stop 2024-04-22T07:04:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf1b723640 1 -- 172.21.15.44:0/1391248818 shutdown_connections 2024-04-22T07:04:57.283 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf1b723640 1 --2- 172.21.15.44:0/1391248818 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf1406c250 0x7faf14154960 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf1b723640 1 --2- 172.21.15.44:0/1391248818 >> v2:172.21.15.70:3300/0 conn(0x7faf14070b30 0x7faf1406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf1b723640 1 -- 172.21.15.44:0/1391248818 >> 172.21.15.44:0/1391248818 conn(0x7faf140cd680 msgr2=0x7faf140cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf1b723640 1 -- 172.21.15.44:0/1391248818 shutdown_connections 2024-04-22T07:04:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.284+0000 7faf1b723640 1 -- 172.21.15.44:0/1391248818 wait complete. 2024-04-22T07:04:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.285+0000 7faf1b723640 1 Processor -- start 2024-04-22T07:04:57.284 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.286+0000 7faf1b723640 1 -- start start 2024-04-22T07:04:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.286+0000 7faf1b723640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf1406c250 0x7faf140bddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.286+0000 7faf1b723640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14070750 0x7faf140c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.286+0000 7faf1b723640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf14070b30 0x7faf14158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.286+0000 7faf1b723640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7faf140735f0 con 0x7faf14070750 2024-04-22T07:04:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.286+0000 7faf1b723640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7faf14073470 con 0x7faf14070b30 2024-04-22T07:04:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.286+0000 7faf1b723640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7faf1415e260 con 0x7faf1406c250 2024-04-22T07:04:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.286+0000 7faf18cbe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14070750 0x7faf140c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.286+0000 7faf18cbe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14070750 0x7faf140c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36816/0 (socket says 172.21.15.44:36816) 2024-04-22T07:04:57.286 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.286+0000 7faf18cbe640 1 -- 172.21.15.44:0/2242551273 learned_addr learned my addr 172.21.15.44:0/2242551273 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf19cc0640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf14070b30 0x7faf14158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf194bf640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf1406c250 0x7faf140bddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf18cbe640 1 -- 172.21.15.44:0/2242551273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf1406c250 msgr2=0x7faf140bddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf18cbe640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf1406c250 0x7faf140bddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf18cbe640 1 -- 172.21.15.44:0/2242551273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf14070b30 msgr2=0x7faf14158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf18cbe640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf14070b30 0x7faf14158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf18cbe640 1 -- 172.21.15.44:0/2242551273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf140be300 con 0x7faf14070750 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf19cc0640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf14070b30 0x7faf14158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf194bf640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf1406c250 0x7faf140bddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf18cbe640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14070750 0x7faf140c8670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faf0800d800 tx=0x7faf0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf11ffb640 1 -- 172.21.15.44:0/2242551273 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08004030 con 0x7faf14070750 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf11ffb640 1 -- 172.21.15.44:0/2242551273 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf080041d0 con 0x7faf14070750 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf1b723640 1 -- 172.21.15.44:0/2242551273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf141f0a90 con 0x7faf14070750 2024-04-22T07:04:57.287 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf11ffb640 1 -- 172.21.15.44:0/2242551273 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0800b6c0 con 0x7faf14070750 2024-04-22T07:04:57.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.287+0000 7faf1b723640 1 -- 172.21.15.44:0/2242551273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141f1050 con 0x7faf14070750 2024-04-22T07:04:57.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.288+0000 7faf1b723640 1 -- 172.21.15.44:0/2242551273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf14069210 con 0x7faf14070750 2024-04-22T07:04:57.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.288+0000 7faf11ffb640 1 -- 172.21.15.44:0/2242551273 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf0800b860 con 0x7faf14070750 2024-04-22T07:04:57.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.289+0000 7faf11ffb640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7faf0006c920 0x7faf0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.289+0000 7faf194bf640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7faf0006c920 0x7faf0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.289+0000 7faf11ffb640 1 -- 172.21.15.44:0/2242551273 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7faf0808be20 con 0x7faf14070750 2024-04-22T07:04:57.288 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.289+0000 7faf194bf640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7faf0006c920 0x7faf0006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faf0c001680 tx=0x7faf0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:57.291 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.293+0000 7faf11ffb640 1 -- 172.21.15.44:0/2242551273 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf0805a1d0 con 0x7faf14070750 2024-04-22T07:04:57.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.477+0000 7faf1b723640 1 -- 172.21.15.44:0/2242551273 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faf1415d350 con 0x7faf14070750 2024-04-22T07:04:57.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.477+0000 7faf11ffb640 1 -- 172.21.15.44:0/2242551273 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7faf08059ad0 con 0x7faf14070750 2024-04-22T07:04:57.477 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T07:04:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.479+0000 7faf1b723640 1 -- 172.21.15.44:0/2242551273 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7faf0006c920 msgr2=0x7faf0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.479+0000 7faf1b723640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7faf0006c920 0x7faf0006ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faf0c001680 tx=0x7faf0c009040 comp rx=0 tx=0).stop 2024-04-22T07:04:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.479+0000 7faf1b723640 1 -- 172.21.15.44:0/2242551273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14070750 msgr2=0x7faf140c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.478 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.479+0000 7faf1b723640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14070750 0x7faf140c8670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faf0800d800 tx=0x7faf0800dcc0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.479+0000 7faf1b723640 1 -- 172.21.15.44:0/2242551273 shutdown_connections 2024-04-22T07:04:57.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.479+0000 7faf1b723640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7faf0006c920 0x7faf0006ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.479+0000 7faf1b723640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf14070b30 0x7faf14158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.479+0000 7faf1b723640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7faf14070750 0x7faf140c8670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.479+0000 7faf1b723640 1 --2- 172.21.15.44:0/2242551273 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7faf1406c250 0x7faf140bddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.479 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.479+0000 7faf1b723640 1 -- 172.21.15.44:0/2242551273 >> 172.21.15.44:0/2242551273 conn(0x7faf140cd680 msgr2=0x7faf1415c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:57.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.480+0000 7faf1b723640 1 -- 172.21.15.44:0/2242551273 shutdown_connections 2024-04-22T07:04:57.480 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.480+0000 7faf1b723640 1 -- 172.21.15.44:0/2242551273 wait complete. 2024-04-22T07:04:57.488 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:04:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.594+0000 7f8168c55640 1 Processor -- start 2024-04-22T07:04:57.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.594+0000 7f8168c55640 1 -- start start 2024-04-22T07:04:57.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8168c55640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8164151de0 0x7f81640cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8168c55640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f816415b950 0x7f81640676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8168c55640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8164067bf0 0x7f816415c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8168c55640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f81640c6770 con 0x7f816415b950 2024-04-22T07:04:57.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8168c55640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f81640c5880 con 0x7f8164067bf0 2024-04-22T07:04:57.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8168c55640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f81640c5b60 con 0x7f8164151de0 2024-04-22T07:04:57.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8161d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f816415b950 0x7f81640676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8161d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f816415b950 0x7f81640676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36828/0 (socket says 172.21.15.44:36828) 2024-04-22T07:04:57.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8161d9b640 1 -- 172.21.15.44:0/3168106907 learned_addr learned my addr 172.21.15.44:0/3168106907 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:57.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8162d9d640 1 --2- 172.21.15.44:0/3168106907 >> v2:172.21.15.70:3300/0 conn(0x7f8164067bf0 0x7f816415c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f816259c640 1 --2- 172.21.15.44:0/3168106907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8164151de0 0x7f81640cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8161d9b640 1 -- 172.21.15.44:0/3168106907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8164151de0 msgr2=0x7f81640cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:57.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8161d9b640 1 --2- 172.21.15.44:0/3168106907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8164151de0 0x7f81640cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8161d9b640 1 -- 172.21.15.44:0/3168106907 >> v2:172.21.15.70:3300/0 conn(0x7f8164067bf0 msgr2=0x7f816415c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:57.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8161d9b640 1 --2- 172.21.15.44:0/3168106907 >> v2:172.21.15.70:3300/0 conn(0x7f8164067bf0 0x7f816415c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.595+0000 7f8161d9b640 1 -- 172.21.15.44:0/3168106907 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81640c5e40 con 0x7f816415b950 2024-04-22T07:04:57.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.596+0000 7f8161d9b640 1 --2- 172.21.15.44:0/3168106907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f816415b950 0x7f81640676b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81640cad50 tx=0x7f815802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e75c8b773c44937 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:57.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.596+0000 7f816159a640 1 -- 172.21.15.44:0/3168106907 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158034070 con 0x7f816415b950 2024-04-22T07:04:57.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.596+0000 7f816159a640 1 -- 172.21.15.44:0/3168106907 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8158030860 con 0x7f816415b950 2024-04-22T07:04:57.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.596+0000 7f816159a640 1 -- 172.21.15.44:0/3168106907 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158030c30 con 0x7f816415b950 2024-04-22T07:04:57.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.597+0000 7f8168c55640 1 -- 172.21.15.44:0/3168106907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f816415b950 msgr2=0x7f81640676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.597+0000 7f8168c55640 1 --2- 172.21.15.44:0/3168106907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f816415b950 0x7f81640676b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f81640cad50 tx=0x7f815802b9a0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.597+0000 7f8168c55640 1 -- 172.21.15.44:0/3168106907 shutdown_connections 2024-04-22T07:04:57.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.597+0000 7f8168c55640 1 --2- 172.21.15.44:0/3168106907 >> v2:172.21.15.70:3300/0 conn(0x7f8164067bf0 0x7f816415c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.597+0000 7f8168c55640 1 --2- 172.21.15.44:0/3168106907 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f816415b950 0x7f81640676b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.597+0000 7f8168c55640 1 --2- 172.21.15.44:0/3168106907 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8164151de0 0x7f81640cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.597+0000 7f8168c55640 1 -- 172.21.15.44:0/3168106907 >> 172.21.15.44:0/3168106907 conn(0x7f8164073da0 msgr2=0x7f81640728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:57.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.597+0000 7f8168c55640 1 -- 172.21.15.44:0/3168106907 shutdown_connections 2024-04-22T07:04:57.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.597+0000 7f8168c55640 1 -- 172.21.15.44:0/3168106907 wait complete. 2024-04-22T07:04:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.598+0000 7f8168c55640 1 Processor -- start 2024-04-22T07:04:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.598+0000 7f8168c55640 1 -- start start 2024-04-22T07:04:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8168c55640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8164067bf0 0x7f816415ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8168c55640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8164151de0 0x7f816415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8168c55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f816415b950 0x7f816415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8168c55640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f816415bd90 con 0x7f8164151de0 2024-04-22T07:04:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8168c55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f81640c6400 con 0x7f816415b950 2024-04-22T07:04:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8168c55640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f816406c770 con 0x7f8164067bf0 2024-04-22T07:04:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8161d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8164151de0 0x7f816415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8161d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8164151de0 0x7f816415dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36834/0 (socket says 172.21.15.44:36834) 2024-04-22T07:04:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8161d9b640 1 -- 172.21.15.44:0/4164897342 learned_addr learned my addr 172.21.15.44:0/4164897342 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:57.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f816259c640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8164067bf0 0x7f816415ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8162d9d640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f816415b950 0x7f816415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8161d9b640 1 -- 172.21.15.44:0/4164897342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8164067bf0 msgr2=0x7f816415ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8161d9b640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8164067bf0 0x7f816415ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8161d9b640 1 -- 172.21.15.44:0/4164897342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f816415b950 msgr2=0x7f816415e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8161d9b640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f816415b950 0x7f816415e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8161d9b640 1 -- 172.21.15.44:0/4164897342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816415a250 con 0x7f8164151de0 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.599+0000 7f8162d9d640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f816415b950 0x7f816415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.600+0000 7f8161d9b640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8164151de0 0x7f816415dc00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f815802b9e0 tx=0x7f8158004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.600+0000 7f8152ffd640 1 -- 172.21.15.44:0/4164897342 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158048070 con 0x7f8164151de0 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.600+0000 7f8152ffd640 1 -- 172.21.15.44:0/4164897342 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8158030660 con 0x7f8164151de0 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.600+0000 7f8152ffd640 1 -- 172.21.15.44:0/4164897342 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158034040 con 0x7f8164151de0 2024-04-22T07:04:57.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.600+0000 7f8168c55640 1 -- 172.21.15.44:0/4164897342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81641e8110 con 0x7f8164151de0 2024-04-22T07:04:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.600+0000 7f8168c55640 1 -- 172.21.15.44:0/4164897342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81641f1490 con 0x7f8164151de0 2024-04-22T07:04:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.601+0000 7f8168c55640 1 -- 172.21.15.44:0/4164897342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8164068b00 con 0x7f8164151de0 2024-04-22T07:04:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.601+0000 7f8152ffd640 1 -- 172.21.15.44:0/4164897342 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f815804b070 con 0x7f8164151de0 2024-04-22T07:04:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.602+0000 7f8152ffd640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f814406c920 0x7f814406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.602+0000 7f8152ffd640 1 -- 172.21.15.44:0/4164897342 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f81580adc70 con 0x7f8164151de0 2024-04-22T07:04:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.602+0000 7f816259c640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f814406c920 0x7f814406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.602+0000 7f816259c640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f814406c920 0x7f814406ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f815c009a10 tx=0x7f815c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:57.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.606+0000 7f8152ffd640 1 -- 172.21.15.44:0/4164897342 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8158030830 con 0x7f8164151de0 2024-04-22T07:04:57.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.788+0000 7f8168c55640 1 -- 172.21.15.44:0/4164897342 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f816414f7a0 con 0x7f8164151de0 2024-04-22T07:04:57.786 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.788+0000 7f8152ffd640 1 -- 172.21.15.44:0/4164897342 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f815807bf10 con 0x7f8164151de0 2024-04-22T07:04:57.787 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T07:04:57.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 -- 172.21.15.44:0/4164897342 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f814406c920 msgr2=0x7f814406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f814406c920 0x7f814406ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f815c009a10 tx=0x7f815c008040 comp rx=0 tx=0).stop 2024-04-22T07:04:57.788 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 -- 172.21.15.44:0/4164897342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8164151de0 msgr2=0x7f816415dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8164151de0 0x7f816415dc00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f815802b9e0 tx=0x7f8158004730 comp rx=0 tx=0).stop 2024-04-22T07:04:57.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 -- 172.21.15.44:0/4164897342 shutdown_connections 2024-04-22T07:04:57.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f814406c920 0x7f814406ede0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f816415b950 0x7f816415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8164151de0 0x7f816415dc00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 --2- 172.21.15.44:0/4164897342 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8164067bf0 0x7f816415ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 -- 172.21.15.44:0/4164897342 >> 172.21.15.44:0/4164897342 conn(0x7f8164073da0 msgr2=0x7f81640cef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:57.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 -- 172.21.15.44:0/4164897342 shutdown_connections 2024-04-22T07:04:57.789 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.790+0000 7f8168c55640 1 -- 172.21.15.44:0/4164897342 wait complete. 2024-04-22T07:04:57.799 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:04:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.905+0000 7f6fbc1c7640 1 Processor -- start 2024-04-22T07:04:57.903 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.905+0000 7f6fbc1c7640 1 -- start start 2024-04-22T07:04:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.906+0000 7f6fbc1c7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fb41552c0 0x7f6fb40b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.906+0000 7f6fbc1c7640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6fb40caec0 0x7f6fb40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.906+0000 7f6fbc1c7640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb41556a0 con 0x7f6fb40cc920 2024-04-22T07:04:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.906+0000 7f6fbc1c7640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f6fb415b160 con 0x7f6fb40caec0 2024-04-22T07:04:57.905 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.906+0000 7f6fbc1c7640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb415b440 con 0x7f6fb41552c0 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.906+0000 7f6fb9762640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f6fb40caec0 0x7f6fb40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.906+0000 7f6fba764640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f6fb40cc920 0x7f6fb40cd310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60728/0 (socket says 172.21.15.44:60728) 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.906+0000 7f6fba764640 1 -- 172.21.15.44:0/3613944139 learned_addr learned my addr 172.21.15.44:0/3613944139 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.907+0000 7f6fb9f63640 1 --2- 172.21.15.44:0/3613944139 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fb41552c0 0x7f6fb40b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.907+0000 7f6fb8f61640 1 -- 172.21.15.44:0/3613944139 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1045720885 0 0) 0x7f6fb41556a0 con 0x7f6fb40cc920 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.907+0000 7f6fb8f61640 1 -- 172.21.15.44:0/3613944139 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f9c003520 con 0x7f6fb40cc920 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.907+0000 7f6fb9762640 1 -- 172.21.15.44:0/3613944139 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fb41552c0 msgr2=0x7f6fb40b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.907+0000 7f6fb9762640 1 --2- 172.21.15.44:0/3613944139 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fb41552c0 0x7f6fb40b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.907+0000 7f6fb9762640 1 -- 172.21.15.44:0/3613944139 >> v1:172.21.15.44:6789/0 conn(0x7f6fb40cc920 legacy=0x7f6fb40cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.907+0000 7f6fb9762640 1 -- 172.21.15.44:0/3613944139 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb415b720 con 0x7f6fb40caec0 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.907+0000 7f6fb9762640 1 --2- 172.21.15.44:0/3613944139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 0x7f6fb40d0a20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80097a0 tx=0x7f6fa802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1269453d30dc8f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fb8f61640 1 -- 172.21.15.44:0/3613944139 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8035020 con 0x7f6fb40caec0 2024-04-22T07:04:57.906 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fb8f61640 1 -- 172.21.15.44:0/3613944139 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa8031830 con 0x7f6fb40caec0 2024-04-22T07:04:57.907 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fb8f61640 1 -- 172.21.15.44:0/3613944139 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8031c00 con 0x7f6fb40caec0 2024-04-22T07:04:57.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/3613944139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 msgr2=0x7f6fb40d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fbc1c7640 1 --2- 172.21.15.44:0/3613944139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 0x7f6fb40d0a20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80097a0 tx=0x7f6fa802b8d0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/3613944139 shutdown_connections 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fbc1c7640 1 --2- 172.21.15.44:0/3613944139 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 0x7f6fb40d0a20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fbc1c7640 1 --2- 172.21.15.44:0/3613944139 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fb41552c0 0x7f6fb40b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/3613944139 >> 172.21.15.44:0/3613944139 conn(0x7f6fb40676b0 msgr2=0x7f6fb4067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/3613944139 shutdown_connections 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.908+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/3613944139 wait complete. 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.909+0000 7f6fbc1c7640 1 Processor -- start 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.909+0000 7f6fbc1c7640 1 -- start start 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.910+0000 7f6fbc1c7640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 0x7f6fb415e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.910+0000 7f6fbc1c7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fb40cc920 0x7f6fb4150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.910+0000 7f6fbc1c7640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fb41552c0 0x7f6fb4150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.910 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.910+0000 7f6fbc1c7640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb40c6a00 con 0x7f6fb40caec0 2024-04-22T07:04:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.910+0000 7f6fbc1c7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb415bce0 con 0x7f6fb40cc920 2024-04-22T07:04:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.910+0000 7f6fbc1c7640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb40cda20 con 0x7f6fb41552c0 2024-04-22T07:04:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.910+0000 7f6fb9f63640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 0x7f6fb415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.910+0000 7f6fb9f63640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 0x7f6fb415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36844/0 (socket says 172.21.15.44:36844) 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.910+0000 7f6fb9f63640 1 -- 172.21.15.44:0/1023669125 learned_addr learned my addr 172.21.15.44:0/1023669125 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.910+0000 7f6fba764640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fb41552c0 0x7f6fb4150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fb9f63640 1 -- 172.21.15.44:0/1023669125 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fb41552c0 msgr2=0x7f6fb4150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fb9f63640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fb41552c0 0x7f6fb4150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fb9762640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fb40cc920 0x7f6fb4150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fb9f63640 1 -- 172.21.15.44:0/1023669125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fb40cc920 msgr2=0x7f6fb4150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fb9f63640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fb40cc920 0x7f6fb4150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fb9f63640 1 -- 172.21.15.44:0/1023669125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb414ec90 con 0x7f6fb40caec0 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fb9f63640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 0x7f6fb415e720 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00d620 tx=0x7f6fac00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fb27fc640 1 -- 172.21.15.44:0/1023669125 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00b620 con 0x7f6fb40caec0 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fb27fc640 1 -- 172.21.15.44:0/1023669125 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fac019070 con 0x7f6fb40caec0 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/1023669125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb414ef70 con 0x7f6fb40caec0 2024-04-22T07:04:57.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fb27fc640 1 -- 172.21.15.44:0/1023669125 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00ba00 con 0x7f6fb40caec0 2024-04-22T07:04:57.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.911+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/1023669125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb414f530 con 0x7f6fb40caec0 2024-04-22T07:04:57.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.912+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/1023669125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb4068b00 con 0x7f6fb40caec0 2024-04-22T07:04:57.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.912+0000 7f6fb27fc640 1 -- 172.21.15.44:0/1023669125 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fac00bba0 con 0x7f6fb40caec0 2024-04-22T07:04:57.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.913+0000 7f6fb27fc640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6fa006c8f0 0x7f6fa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:57.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.913+0000 7f6fb27fc640 1 -- 172.21.15.44:0/1023669125 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f6fac08c050 con 0x7f6fb40caec0 2024-04-22T07:04:57.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.913+0000 7f6fb9762640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6fa006c8f0 0x7f6fa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:57.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.913+0000 7f6fb9762640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6fa006c8f0 0x7f6fa006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8036000 tx=0x7f6fa8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:57.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:57.917+0000 7f6fb27fc640 1 -- 172.21.15.44:0/1023669125 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fac05a400 con 0x7f6fb40caec0 2024-04-22T07:04:58.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.106+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/1023669125 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6fb415f3a0 con 0x7f6fb40caec0 2024-04-22T07:04:58.104 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.107+0000 7f6fb27fc640 1 -- 172.21.15.44:0/1023669125 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f6fac059d00 con 0x7f6fb40caec0 2024-04-22T07:04:58.105 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T07:04:58.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/1023669125 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6fa006c8f0 msgr2=0x7f6fa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6fa006c8f0 0x7f6fa006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8036000 tx=0x7f6fa8008030 comp rx=0 tx=0).stop 2024-04-22T07:04:58.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/1023669125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 msgr2=0x7f6fb415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 0x7f6fb415e720 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00d620 tx=0x7f6fac00dae0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/1023669125 shutdown_connections 2024-04-22T07:04:58.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f6fa006c8f0 0x7f6fa006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f6fb41552c0 0x7f6fb4150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fb40cc920 0x7f6fb4150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 --2- 172.21.15.44:0/1023669125 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f6fb40caec0 0x7f6fb415e720 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.107 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/1023669125 >> 172.21.15.44:0/1023669125 conn(0x7f6fb40676b0 msgr2=0x7f6fb4067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:58.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/1023669125 shutdown_connections 2024-04-22T07:04:58.108 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.109+0000 7f6fbc1c7640 1 -- 172.21.15.44:0/1023669125 wait complete. 2024-04-22T07:04:58.117 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.221+0000 7f7087784640 1 Processor -- start 2024-04-22T07:04:58.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.221+0000 7f7087784640 1 -- start start 2024-04-22T07:04:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.222+0000 7f7087784640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7080154f70 0x7f708015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.222+0000 7f7087784640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cd6f0 0x7f70800c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.222+0000 7f7087784640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f70800cb3e0 0x7f70800cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.222+0000 7f7087784640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f70800cd260 con 0x7f70800cb3e0 2024-04-22T07:04:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.222+0000 7f7087784640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f70800c5d40 con 0x7f7080154f70 2024-04-22T07:04:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.222+0000 7f7087784640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f708015ee90 con 0x7f70800cd6f0 2024-04-22T07:04:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.222+0000 7f7085d21640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f70800cb3e0 0x7f70800cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.222+0000 7f7085d21640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f70800cb3e0 0x7f70800cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36860/0 (socket says 172.21.15.44:36860) 2024-04-22T07:04:58.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.222+0000 7f7085d21640 1 -- 172.21.15.44:0/2430262906 learned_addr learned my addr 172.21.15.44:0/2430262906 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.223+0000 7f7085520640 1 --2- 172.21.15.44:0/2430262906 >> v2:172.21.15.70:3300/0 conn(0x7f7080154f70 0x7f708015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.223+0000 7f7084d1f640 1 --2- 172.21.15.44:0/2430262906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cd6f0 0x7f70800c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.223+0000 7f7085d21640 1 -- 172.21.15.44:0/2430262906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cd6f0 msgr2=0x7f70800c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.223+0000 7f7085d21640 1 --2- 172.21.15.44:0/2430262906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cd6f0 0x7f70800c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.223+0000 7f7085d21640 1 -- 172.21.15.44:0/2430262906 >> v2:172.21.15.70:3300/0 conn(0x7f7080154f70 msgr2=0x7f708015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.223+0000 7f7085d21640 1 --2- 172.21.15.44:0/2430262906 >> v2:172.21.15.70:3300/0 conn(0x7f7080154f70 0x7f708015ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.223+0000 7f7085d21640 1 -- 172.21.15.44:0/2430262906 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708015f170 con 0x7f70800cb3e0 2024-04-22T07:04:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.223+0000 7f7084d1f640 1 --2- 172.21.15.44:0/2430262906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cd6f0 0x7f70800c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:58.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.223+0000 7f7085d21640 1 --2- 172.21.15.44:0/2430262906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cb3e0 0x7f70800cb890 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70800cafe0 tx=0x7f707802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbfdaf4ba81c093 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:58.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7077fff640 1 -- 172.21.15.44:0/2430262906 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707802cb00 con 0x7f70800cb3e0 2024-04-22T07:04:58.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7077fff640 1 -- 172.21.15.44:0/2430262906 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707802cca0 con 0x7f70800cb3e0 2024-04-22T07:04:58.225 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7077fff640 1 -- 172.21.15.44:0/2430262906 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7078034610 con 0x7f70800cb3e0 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7087784640 1 -- 172.21.15.44:0/2430262906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cb3e0 msgr2=0x7f70800cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7087784640 1 --2- 172.21.15.44:0/2430262906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cb3e0 0x7f70800cb890 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f70800cafe0 tx=0x7f707802bf60 comp rx=0 tx=0).stop 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7087784640 1 -- 172.21.15.44:0/2430262906 shutdown_connections 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7087784640 1 --2- 172.21.15.44:0/2430262906 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cb3e0 0x7f70800cb890 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7087784640 1 --2- 172.21.15.44:0/2430262906 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cd6f0 0x7f70800c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7087784640 1 --2- 172.21.15.44:0/2430262906 >> v2:172.21.15.70:3300/0 conn(0x7f7080154f70 0x7f708015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7087784640 1 -- 172.21.15.44:0/2430262906 >> 172.21.15.44:0/2430262906 conn(0x7f7080067d40 msgr2=0x7f708006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7087784640 1 -- 172.21.15.44:0/2430262906 shutdown_connections 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.224+0000 7f7087784640 1 -- 172.21.15.44:0/2430262906 wait complete. 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.225+0000 7f7087784640 1 Processor -- start 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.225+0000 7f7087784640 1 -- start start 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7087784640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cb3e0 0x7f70801f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7087784640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cd6f0 0x7f708015a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7087784640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7080154f70 0x7f70801f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7087784640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f70800ccef0 con 0x7f70800cd6f0 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7087784640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f70800ccc10 con 0x7f7080154f70 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7087784640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7080155350 con 0x7f70800cb3e0 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7084d1f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cd6f0 0x7f708015a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7084d1f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cd6f0 0x7f708015a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36870/0 (socket says 172.21.15.44:36870) 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7084d1f640 1 -- 172.21.15.44:0/1182650451 learned_addr learned my addr 172.21.15.44:0/1182650451 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7085520640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cb3e0 0x7f70801f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.226+0000 7f7085d21640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7080154f70 0x7f70801f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7084d1f640 1 -- 172.21.15.44:0/1182650451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cb3e0 msgr2=0x7f70801f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7084d1f640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cb3e0 0x7f70801f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7084d1f640 1 -- 172.21.15.44:0/1182650451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7080154f70 msgr2=0x7f70801f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7084d1f640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7080154f70 0x7f70801f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7084d1f640 1 -- 172.21.15.44:0/1182650451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70801f32b0 con 0x7f70800cd6f0 2024-04-22T07:04:58.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7085520640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cb3e0 0x7f70801f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:58.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7085d21640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7080154f70 0x7f70801f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:58.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7084d1f640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cd6f0 0x7f708015a400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f707000c7c0 tx=0x7f707000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:58.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7075ffb640 1 -- 172.21.15.44:0/1182650451 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070015070 con 0x7f70800cd6f0 2024-04-22T07:04:58.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7075ffb640 1 -- 172.21.15.44:0/1182650451 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7070007a30 con 0x7f70800cd6f0 2024-04-22T07:04:58.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7087784640 1 -- 172.21.15.44:0/1182650451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708015a940 con 0x7f70800cd6f0 2024-04-22T07:04:58.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7075ffb640 1 -- 172.21.15.44:0/1182650451 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070007e00 con 0x7f70800cd6f0 2024-04-22T07:04:58.228 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.227+0000 7f7087784640 1 -- 172.21.15.44:0/1182650451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708015abd0 con 0x7f70800cd6f0 2024-04-22T07:04:58.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.228+0000 7f7087784640 1 -- 172.21.15.44:0/1182650451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70800c5580 con 0x7f70800cd6f0 2024-04-22T07:04:58.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.228+0000 7f7075ffb640 1 -- 172.21.15.44:0/1182650451 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7070018070 con 0x7f70800cd6f0 2024-04-22T07:04:58.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.229+0000 7f7075ffb640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f706406c8f0 0x7f706406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.229+0000 7f7075ffb640 1 -- 172.21.15.44:0/1182650451 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f707008bb30 con 0x7f70800cd6f0 2024-04-22T07:04:58.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.229+0000 7f7085520640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f706406c8f0 0x7f706406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.229 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.229+0000 7f7085520640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f706406c8f0 0x7f706406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f707c006fd0 tx=0x7f707c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:58.231 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.233+0000 7f7075ffb640 1 -- 172.21.15.44:0/1182650451 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70800c5580 con 0x7f70800cd6f0 2024-04-22T07:04:58.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.463+0000 7f7087784640 1 -- 172.21.15.44:0/1182650451 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70800d1340 con 0x7f70800cd6f0 2024-04-22T07:04:58.461 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.464+0000 7f7075ffb640 1 -- 172.21.15.44:0/1182650451 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6908 (secure 0 0 0) 0x7f70800d1340 con 0x7f70800cd6f0 2024-04-22T07:04:58.461 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:58.462 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:04:44.948538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4324,"mds_1":4303,"mds_2":4300,"mds_3":4278,"mds_4":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"b","rank":3,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4324,"qdb_cluster":[4275,4278,4300,4303,4324]},"id":1}]} 2024-04-22T07:04:58.462 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-04-22T07:04:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 -- 172.21.15.44:0/1182650451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f706406c8f0 msgr2=0x7f706406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f706406c8f0 0x7f706406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f707c006fd0 tx=0x7f707c008040 comp rx=0 tx=0).stop 2024-04-22T07:04:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 -- 172.21.15.44:0/1182650451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cd6f0 msgr2=0x7f708015a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cd6f0 0x7f708015a400 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f707000c7c0 tx=0x7f707000cc80 comp rx=0 tx=0).stop 2024-04-22T07:04:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 -- 172.21.15.44:0/1182650451 shutdown_connections 2024-04-22T07:04:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f706406c8f0 0x7f706406edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7080154f70 0x7f70801f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f70800cd6f0 0x7f708015a400 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 --2- 172.21.15.44:0/1182650451 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f70800cb3e0 0x7f70801f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 -- 172.21.15.44:0/1182650451 >> 172.21.15.44:0/1182650451 conn(0x7f7080067d40 msgr2=0x7f708015e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:58.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 -- 172.21.15.44:0/1182650451 shutdown_connections 2024-04-22T07:04:58.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.466+0000 7f7087784640 1 -- 172.21.15.44:0/1182650451 wait complete. 2024-04-22T07:04:58.475 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:04:58.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.577+0000 7f5372952640 1 Processor -- start 2024-04-22T07:04:58.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.577+0000 7f5372952640 1 -- start start 2024-04-22T07:04:58.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.578+0000 7f5372952640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f536c155120 0x7f536c0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.578+0000 7f5372952640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f536c155500 con 0x7f536c15e8b0 2024-04-22T07:04:58.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.578+0000 7f5372952640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f536c072f90 con 0x7f536c0caec0 2024-04-22T07:04:58.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.578+0000 7f5372952640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f536c15b0d0 con 0x7f536c155120 2024-04-22T07:04:58.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.578+0000 7f536bfff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f536c155120 0x7f536c0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.578+0000 7f5370eef640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f536c15e8b0 0x7f536c15ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60732/0 (socket says 172.21.15.44:60732) 2024-04-22T07:04:58.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.578+0000 7f5370eef640 1 -- 172.21.15.44:0/3493588279 learned_addr learned my addr 172.21.15.44:0/3493588279 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:58.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.578+0000 7f536bfff640 1 -- 172.21.15.44:0/3493588279 >> v1:172.21.15.70:6789/0 conn(0x7f536c0caec0 legacy=0x7f536c0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:58.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.578+0000 7f536bfff640 1 -- 172.21.15.44:0/3493588279 >> v1:172.21.15.44:6789/0 conn(0x7f536c15e8b0 legacy=0x7f536c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.579+0000 7f536affd640 1 -- 172.21.15.44:0/3493588279 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3887705795 0 0) 0x7f536c155500 con 0x7f536c15e8b0 2024-04-22T07:04:58.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.579+0000 7f536bfff640 1 -- 172.21.15.44:0/3493588279 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c15b3b0 con 0x7f536c155120 2024-04-22T07:04:58.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.579+0000 7f536bfff640 1 --2- 172.21.15.44:0/3493588279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c155120 0x7f536c0c6380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f53600097a0 tx=0x7f536002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ce7bd2030ce8bb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.579+0000 7f536affd640 1 -- 172.21.15.44:0/3493588279 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360034070 con 0x7f536c155120 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.579+0000 7f536affd640 1 -- 172.21.15.44:0/3493588279 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5360030610 con 0x7f536c155120 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.579+0000 7f536affd640 1 -- 172.21.15.44:0/3493588279 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53600309e0 con 0x7f536c155120 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.580+0000 7f5372952640 1 -- 172.21.15.44:0/3493588279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c155120 msgr2=0x7f536c0c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.580+0000 7f5372952640 1 --2- 172.21.15.44:0/3493588279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c155120 0x7f536c0c6380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f53600097a0 tx=0x7f536002b960 comp rx=0 tx=0).stop 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.580+0000 7f5372952640 1 -- 172.21.15.44:0/3493588279 shutdown_connections 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.580+0000 7f5372952640 1 --2- 172.21.15.44:0/3493588279 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c155120 0x7f536c0c6380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.580+0000 7f5372952640 1 -- 172.21.15.44:0/3493588279 >> 172.21.15.44:0/3493588279 conn(0x7f536c0676b0 msgr2=0x7f536c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.580+0000 7f5372952640 1 -- 172.21.15.44:0/3493588279 shutdown_connections 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.580+0000 7f5372952640 1 -- 172.21.15.44:0/3493588279 wait complete. 2024-04-22T07:04:58.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.581+0000 7f5372952640 1 Processor -- start 2024-04-22T07:04:58.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.581+0000 7f5372952640 1 -- start start 2024-04-22T07:04:58.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f5372952640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c0caec0 0x7f536c153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f5372952640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f536c155120 0x7f536c1f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f5372952640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c15e8b0 0x7f536c147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f5372952640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f536c15bc50 con 0x7f536c0caec0 2024-04-22T07:04:58.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f5372952640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f536c15b970 con 0x7f536c155120 2024-04-22T07:04:58.581 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f5372952640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f536c15e390 con 0x7f536c15e8b0 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f536bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c0caec0 0x7f536c153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f536bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c0caec0 0x7f536c153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36882/0 (socket says 172.21.15.44:36882) 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f536bfff640 1 -- 172.21.15.44:0/270033307 learned_addr learned my addr 172.21.15.44:0/270033307 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f5370eef640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c15e8b0 0x7f536c147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.582+0000 7f536b7fe640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f536c155120 0x7f536c1f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f536bfff640 1 -- 172.21.15.44:0/270033307 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c15e8b0 msgr2=0x7f536c147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f536bfff640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c15e8b0 0x7f536c147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f536bfff640 1 -- 172.21.15.44:0/270033307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f536c155120 msgr2=0x7f536c1f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f536bfff640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f536c155120 0x7f536c1f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f536bfff640 1 -- 172.21.15.44:0/270033307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c1f5290 con 0x7f536c0caec0 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f536bfff640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c0caec0 0x7f536c153dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f53600097a0 tx=0x7f53600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f5368ff9640 1 -- 172.21.15.44:0/270033307 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360034040 con 0x7f536c0caec0 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f5368ff9640 1 -- 172.21.15.44:0/270033307 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5360004030 con 0x7f536c0caec0 2024-04-22T07:04:58.582 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f5372952640 1 -- 172.21.15.44:0/270033307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536c145f40 con 0x7f536c0caec0 2024-04-22T07:04:58.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f5368ff9640 1 -- 172.21.15.44:0/270033307 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360004420 con 0x7f536c0caec0 2024-04-22T07:04:58.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.583+0000 7f5372952640 1 -- 172.21.15.44:0/270033307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536c146500 con 0x7f536c0caec0 2024-04-22T07:04:58.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.584+0000 7f5372952640 1 -- 172.21.15.44:0/270033307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536c072890 con 0x7f536c0caec0 2024-04-22T07:04:58.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.584+0000 7f5368ff9640 1 -- 172.21.15.44:0/270033307 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5360037070 con 0x7f536c0caec0 2024-04-22T07:04:58.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.585+0000 7f5368ff9640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f535806c920 0x7f535806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.585+0000 7f5368ff9640 1 -- 172.21.15.44:0/270033307 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f53600ad7c0 con 0x7f536c0caec0 2024-04-22T07:04:58.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.585+0000 7f536b7fe640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f535806c920 0x7f535806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.585+0000 7f536b7fe640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f535806c920 0x7f535806ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f536c0c6500 tx=0x7f535c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:58.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.589+0000 7f5368ff9640 1 -- 172.21.15.44:0/270033307 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f536c072890 con 0x7f536c0caec0 2024-04-22T07:04:58.775 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.778+0000 7f5372952640 1 -- 172.21.15.44:0/270033307 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f536c04fdc0 con 0x7f536c0caec0 2024-04-22T07:04:58.776 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.779+0000 7f5368ff9640 1 -- 172.21.15.44:0/270033307 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16807 (secure 0 0 0) 0x7f536007b5c0 con 0x7f536c0caec0 2024-04-22T07:04:58.777 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:58.777 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":25,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:04:54.977439+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:04:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 -- 172.21.15.44:0/270033307 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f535806c920 msgr2=0x7f535806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f535806c920 0x7f535806ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f536c0c6500 tx=0x7f535c009040 comp rx=0 tx=0).stop 2024-04-22T07:04:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 -- 172.21.15.44:0/270033307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c0caec0 msgr2=0x7f536c153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c0caec0 0x7f536c153dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f53600097a0 tx=0x7f53600046c0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 -- 172.21.15.44:0/270033307 shutdown_connections 2024-04-22T07:04:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f535806c920 0x7f535806ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f536c15e8b0 0x7f536c147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.779 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f536c155120 0x7f536c1f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 --2- 172.21.15.44:0/270033307 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f536c0caec0 0x7f536c153dc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 -- 172.21.15.44:0/270033307 >> 172.21.15.44:0/270033307 conn(0x7f536c0676b0 msgr2=0x7f536c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:58.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 -- 172.21.15.44:0/270033307 shutdown_connections 2024-04-22T07:04:58.780 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.781+0000 7f5372952640 1 -- 172.21.15.44:0/270033307 wait complete. 2024-04-22T07:04:58.790 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 572406 2024-04-22T07:04:58.790 DEBUG:tasks.quiescer:Yielding 2024-04-22T07:04:58.790 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T07:04:58.803 INFO:tasks.workunit:Pulling workunits from ref 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:04:58.803 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T07:04:58.803 INFO:tasks.workunit:timeout=6h 2024-04-22T07:04:58.803 INFO:tasks.workunit:cleanup=True 2024-04-22T07:04:58.804 DEBUG:teuthology.orchestra.run.smithi044:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:58.804 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:04:58.820 INFO:teuthology.orchestra.run.smithi044.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-22T07:04:58.821 INFO:teuthology.orchestra.run.smithi044.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-04-22T07:04:58.821 INFO:teuthology.orchestra.run.smithi044.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T07:04:58.821 INFO:teuthology.orchestra.run.smithi044.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-22T07:04:58.821 INFO:teuthology.orchestra.run.smithi044.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-22T07:04:58.821 INFO:teuthology.orchestra.run.smithi044.stdout:Modify: 2024-04-22 07:04:18.793686230 +0000 2024-04-22T07:04:58.821 INFO:teuthology.orchestra.run.smithi044.stdout:Change: 2024-04-22 07:04:52.877179201 +0000 2024-04-22T07:04:58.821 INFO:teuthology.orchestra.run.smithi044.stdout: Birth: - 2024-04-22T07:04:58.821 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-22T07:04:58.821 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-22T07:04:58.891 DEBUG:teuthology.orchestra.run.smithi044:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 86f7587a5a09af35f0895e1a2d08527638fae697 2024-04-22T07:04:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.909+0000 7f8112e2e640 1 Processor -- start 2024-04-22T07:04:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.909+0000 7f8112e2e640 1 -- start start 2024-04-22T07:04:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.910+0000 7f8112e2e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 0x7f810c15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.910+0000 7f8112e2e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f810c15ee90 con 0x7f810c0cb3e0 2024-04-22T07:04:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.910+0000 7f8112e2e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f810c15f170 con 0x7f810c154f70 2024-04-22T07:04:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.910+0000 7f8112e2e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f810c0cd260 con 0x7f810c0cd6f0 2024-04-22T07:04:58.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.910+0000 7f81113cb640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f810c0cb3e0 0x7f810c15e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60742/0 (socket says 172.21.15.44:60742) 2024-04-22T07:04:58.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.910+0000 7f81113cb640 1 -- 172.21.15.44:0/1710060442 learned_addr learned my addr 172.21.15.44:0/1710060442 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:04:58.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.910+0000 7f810bfff640 1 --2- 172.21.15.44:0/1710060442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 0x7f810c15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.910+0000 7f810b7fe640 1 -- 172.21.15.44:0/1710060442 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3648906784 0 0) 0x7f810c15ee90 con 0x7f810c0cb3e0 2024-04-22T07:04:58.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.911+0000 7f810b7fe640 1 -- 172.21.15.44:0/1710060442 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80f8003520 con 0x7f810c0cb3e0 2024-04-22T07:04:58.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.911+0000 7f810b7fe640 1 -- 172.21.15.44:0/1710060442 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1912904220 0 0) 0x7f810c15f170 con 0x7f810c154f70 2024-04-22T07:04:58.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.911+0000 7f810b7fe640 1 -- 172.21.15.44:0/1710060442 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f810c15ee90 con 0x7f810c154f70 2024-04-22T07:04:58.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.911+0000 7f810bfff640 1 -- 172.21.15.44:0/1710060442 >> v1:172.21.15.70:6789/0 conn(0x7f810c154f70 legacy=0x7f810c0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.911+0000 7f810bfff640 1 -- 172.21.15.44:0/1710060442 >> v1:172.21.15.44:6789/0 conn(0x7f810c0cb3e0 legacy=0x7f810c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.911+0000 7f810b7fe640 1 -- 172.21.15.44:0/1710060442 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 654837574 0 0) 0x7f80f8003520 con 0x7f810c0cb3e0 2024-04-22T07:04:58.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.911+0000 7f810bfff640 1 -- 172.21.15.44:0/1710060442 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c0cc930 con 0x7f810c0cd6f0 2024-04-22T07:04:58.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f810bfff640 1 --2- 172.21.15.44:0/1710060442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 0x7f810c15ad30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0097d0 tx=0x7f80fc02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac4efdee3adc6c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f810b7fe640 1 -- 172.21.15.44:0/1710060442 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4001670 con 0x7f810c0cd6f0 2024-04-22T07:04:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f810b7fe640 1 -- 172.21.15.44:0/1710060442 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80fc037070 con 0x7f810c0cd6f0 2024-04-22T07:04:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f810b7fe640 1 -- 172.21.15.44:0/1710060442 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0317a0 con 0x7f810c0cd6f0 2024-04-22T07:04:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f8112e2e640 1 -- 172.21.15.44:0/1710060442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 msgr2=0x7f810c15ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f8112e2e640 1 --2- 172.21.15.44:0/1710060442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 0x7f810c15ad30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0097d0 tx=0x7f80fc02b8a0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f8112e2e640 1 -- 172.21.15.44:0/1710060442 shutdown_connections 2024-04-22T07:04:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f8112e2e640 1 --2- 172.21.15.44:0/1710060442 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 0x7f810c15ad30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f8112e2e640 1 -- 172.21.15.44:0/1710060442 >> 172.21.15.44:0/1710060442 conn(0x7f810c067d40 msgr2=0x7f810c06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f8112e2e640 1 -- 172.21.15.44:0/1710060442 shutdown_connections 2024-04-22T07:04:58.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.912+0000 7f8112e2e640 1 -- 172.21.15.44:0/1710060442 wait complete. 2024-04-22T07:04:58.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.913+0000 7f8112e2e640 1 Processor -- start 2024-04-22T07:04:58.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.914+0000 7f8112e2e640 1 -- start start 2024-04-22T07:04:58.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.914+0000 7f8112e2e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f810c0cb3e0 0x7f810c14b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.914 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.914+0000 7f8112e2e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 0x7f810c157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.914+0000 7f8112e2e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f810c154f70 0x7f810c0cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.914+0000 7f8112e2e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f810c155350 con 0x7f810c154f70 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.914+0000 7f8112e2e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f810c0ccef0 con 0x7f810c0cb3e0 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.914+0000 7f8112e2e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f810c0c91f0 con 0x7f810c0cd6f0 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81113cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f810c154f70 0x7f810c0cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81113cb640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f810c154f70 0x7f810c0cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36892/0 (socket says 172.21.15.44:36892) 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81113cb640 1 -- 172.21.15.44:0/1491078889 learned_addr learned my addr 172.21.15.44:0/1491078889 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f810bfff640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 0x7f810c157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f8110bca640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f810c0cb3e0 0x7f810c14b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81113cb640 1 -- 172.21.15.44:0/1491078889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 msgr2=0x7f810c157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81113cb640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 0x7f810c157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81113cb640 1 -- 172.21.15.44:0/1491078889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f810c0cb3e0 msgr2=0x7f810c14b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:58.915 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81113cb640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f810c0cb3e0 0x7f810c14b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81113cb640 1 -- 172.21.15.44:0/1491078889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c1f1b40 con 0x7f810c154f70 2024-04-22T07:04:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f8110bca640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f810c0cb3e0 0x7f810c14b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f810bfff640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 0x7f810c157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:04:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81113cb640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f810c154f70 0x7f810c0cebc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f80f400c550 tx=0x7f80f400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81097fa640 1 -- 172.21.15.44:0/1491078889 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4005ce0 con 0x7f810c154f70 2024-04-22T07:04:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81097fa640 1 -- 172.21.15.44:0/1491078889 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f4007070 con 0x7f810c154f70 2024-04-22T07:04:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f8112e2e640 1 -- 172.21.15.44:0/1491078889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810c1f0fc0 con 0x7f810c154f70 2024-04-22T07:04:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.915+0000 7f81097fa640 1 -- 172.21.15.44:0/1491078889 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4007460 con 0x7f810c154f70 2024-04-22T07:04:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.916+0000 7f8112e2e640 1 -- 172.21.15.44:0/1491078889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810c1f1580 con 0x7f810c154f70 2024-04-22T07:04:58.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.917+0000 7f8112e2e640 1 -- 172.21.15.44:0/1491078889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810c0cc500 con 0x7f810c154f70 2024-04-22T07:04:58.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.917+0000 7f81097fa640 1 -- 172.21.15.44:0/1491078889 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80f4007600 con 0x7f810c154f70 2024-04-22T07:04:58.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.918+0000 7f81097fa640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80ec06c8f0 0x7f80ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:58.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.918+0000 7f81097fa640 1 -- 172.21.15.44:0/1491078889 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f80f408b860 con 0x7f810c154f70 2024-04-22T07:04:58.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.918+0000 7f8110bca640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80ec06c8f0 0x7f80ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:58.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.918+0000 7f8110bca640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80ec06c8f0 0x7f80ec06edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8104001560 tx=0x7f8104009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:58.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:58.922+0000 7f81097fa640 1 -- 172.21.15.44:0/1491078889 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80f4059c10 con 0x7f810c154f70 2024-04-22T07:04:58.923 INFO:tasks.workunit.client.0.smithi044.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T07:04:59.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.154+0000 7f8112e2e640 1 -- 172.21.15.44:0/1491078889 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f810c04fdc0 con 0x7f810c154f70 2024-04-22T07:04:59.152 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.155+0000 7f81097fa640 1 -- 172.21.15.44:0/1491078889 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6908 (secure 0 0 0) 0x7f80f4059510 con 0x7f810c154f70 2024-04-22T07:04:59.153 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:04:59.153 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:04:44.948538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4324,"mds_1":4303,"mds_2":4300,"mds_3":4278,"mds_4":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"b","rank":3,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4324,"qdb_cluster":[4275,4278,4300,4303,4324]},"id":1}]} 2024-04-22T07:04:59.153 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 -- 172.21.15.44:0/1491078889 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80ec06c8f0 msgr2=0x7f80ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80ec06c8f0 0x7f80ec06edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8104001560 tx=0x7f8104009040 comp rx=0 tx=0).stop 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 -- 172.21.15.44:0/1491078889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f810c154f70 msgr2=0x7f810c0cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f810c154f70 0x7f810c0cebc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f80f400c550 tx=0x7f80f400c950 comp rx=0 tx=0).stop 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 -- 172.21.15.44:0/1491078889 shutdown_connections 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f80ec06c8f0 0x7f80ec06edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f810c0cd6f0 0x7f810c157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f810c154f70 0x7f810c0cebc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 --2- 172.21.15.44:0/1491078889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f810c0cb3e0 0x7f810c14b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 -- 172.21.15.44:0/1491078889 >> 172.21.15.44:0/1491078889 conn(0x7f810c067d40 msgr2=0x7f810c0b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 -- 172.21.15.44:0/1491078889 shutdown_connections 2024-04-22T07:04:59.155 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.157+0000 7f8112e2e640 1 -- 172.21.15.44:0/1491078889 wait complete. 2024-04-22T07:04:59.769 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-22T07:04:13.144503+0000', 'modified': '2024-04-22T07:04:44.948538+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4324, 'mds_1': 4303, 'mds_2': 4300, 'mds_3': 4278, 'mds_4': 4275}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4275': {'gid': 4275, 'name': 'd', 'rank': 4, 'incarnation': 16, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.70:6834/1969018412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 1969018412}, {'type': 'v1', 'addr': '172.21.15.70:6834', 'nonce': 1969018412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4278': {'gid': 4278, 'name': 'b', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.70:6835/4058009549', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6833', 'nonce': 4058009549}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 4058009549}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4300': {'gid': 4300, 'name': 'c', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.44:6838/1520421043', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6836', 'nonce': 1520421043}, {'type': 'v1', 'addr': '172.21.15.44:6838', 'nonce': 1520421043}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4303': {'gid': 4303, 'name': 'e', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.44:6839/1911557586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.44:6837', 'nonce': 1911557586}, {'type': 'v1', 'addr': '172.21.15.44:6839', 'nonce': 1911557586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4324': {'gid': 4324, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.70:6837/2872131879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6836', 'nonce': 2872131879}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 2872131879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4324, 'qdb_cluster': [4275, 4278, 4300, 4303, 4324]} 2024-04-22T07:04:59.769 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T07:04:59.769 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-22T07:04:59.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.877+0000 7f14fd7f4640 1 Processor -- start 2024-04-22T07:04:59.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.877+0000 7f14fd7f4640 1 -- start start 2024-04-22T07:04:59.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14fd7f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 0x7f14f80c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14fd7f4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f14f815e8b0 0x7f14f815ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14fd7f4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f8155500 con 0x7f14f80caec0 2024-04-22T07:04:59.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14fd7f4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f14f815f260 con 0x7f14f815e8b0 2024-04-22T07:04:59.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14fd7f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f14f8072f90 con 0x7f14f8155120 2024-04-22T07:04:59.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14f77fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f14f815e8b0 0x7f14f815ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14f77fe640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f14f815e8b0 0x7f14f815ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36906/0 (socket says 172.21.15.44:36906) 2024-04-22T07:04:59.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14f77fe640 1 -- 172.21.15.44:0/3825393352 learned_addr learned my addr 172.21.15.44:0/3825393352 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:59.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/3825393352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 0x7f14f80c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14f77fe640 1 -- 172.21.15.44:0/3825393352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 msgr2=0x7f14f80c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:59.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14f77fe640 1 --2- 172.21.15.44:0/3825393352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 0x7f14f80c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14f77fe640 1 -- 172.21.15.44:0/3825393352 >> v1:172.21.15.70:6789/0 conn(0x7f14f80caec0 legacy=0x7f14f80d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14f77fe640 1 -- 172.21.15.44:0/3825393352 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f815b0d0 con 0x7f14f815e8b0 2024-04-22T07:04:59.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.878+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/3825393352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 0x7f14f80c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:59.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.879+0000 7f14f77fe640 1 --2- 172.21.15.44:0/3825393352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f815e8b0 0x7f14f815ed20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009e50 tx=0x7f14e802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54d95c8fc15ddda server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:59.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.879+0000 7f14f5ffb640 1 -- 172.21.15.44:0/3825393352 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e802cb00 con 0x7f14f815e8b0 2024-04-22T07:04:59.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.879+0000 7f14f5ffb640 1 -- 172.21.15.44:0/3825393352 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e802cca0 con 0x7f14f815e8b0 2024-04-22T07:04:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.879+0000 7f14f5ffb640 1 -- 172.21.15.44:0/3825393352 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8034610 con 0x7f14f815e8b0 2024-04-22T07:04:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.880+0000 7f14fd7f4640 1 -- 172.21.15.44:0/3825393352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f815e8b0 msgr2=0x7f14f815ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.880+0000 7f14fd7f4640 1 --2- 172.21.15.44:0/3825393352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f815e8b0 0x7f14f815ed20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009e50 tx=0x7f14e802bf60 comp rx=0 tx=0).stop 2024-04-22T07:04:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.880+0000 7f14fd7f4640 1 -- 172.21.15.44:0/3825393352 shutdown_connections 2024-04-22T07:04:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.880+0000 7f14fd7f4640 1 --2- 172.21.15.44:0/3825393352 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f815e8b0 0x7f14f815ed20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.880+0000 7f14fd7f4640 1 --2- 172.21.15.44:0/3825393352 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 0x7f14f80c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.880+0000 7f14fd7f4640 1 -- 172.21.15.44:0/3825393352 >> 172.21.15.44:0/3825393352 conn(0x7f14f80676b0 msgr2=0x7f14f8067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.880+0000 7f14fd7f4640 1 -- 172.21.15.44:0/3825393352 shutdown_connections 2024-04-22T07:04:59.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.880+0000 7f14fd7f4640 1 -- 172.21.15.44:0/3825393352 wait complete. 2024-04-22T07:04:59.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.881+0000 7f14fd7f4640 1 Processor -- start 2024-04-22T07:04:59.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.881+0000 7f14fd7f4640 1 -- start start 2024-04-22T07:04:59.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14fd7f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80caec0 0x7f14f8148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14fd7f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 0x7f14f81f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14fd7f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f815e8b0 0x7f14f814b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.881 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14fd7f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14f815b970 con 0x7f14f80caec0 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14fd7f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f14f815b690 con 0x7f14f815e8b0 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14fd7f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f14f815bc50 con 0x7f14f8155120 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80caec0 0x7f14f8148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14f6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80caec0 0x7f14f8148c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36922/0 (socket says 172.21.15.44:36922) 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14f6ffd640 1 -- 172.21.15.44:0/643124558 learned_addr learned my addr 172.21.15.44:0/643124558 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14f77fe640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f815e8b0 0x7f14f814b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14f67fc640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 0x7f14f81f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14f6ffd640 1 -- 172.21.15.44:0/643124558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 msgr2=0x7f14f81f0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 0x7f14f81f0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14f6ffd640 1 -- 172.21.15.44:0/643124558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f815e8b0 msgr2=0x7f14f814b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f815e8b0 0x7f14f814b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.882+0000 7f14f6ffd640 1 -- 172.21.15.44:0/643124558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f815bf60 con 0x7f14f80caec0 2024-04-22T07:04:59.882 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.883+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80caec0 0x7f14f8148c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f14f000e7e0 tx=0x7f14f000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:59.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.883+0000 7f14ef7fe640 1 -- 172.21.15.44:0/643124558 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f000f040 con 0x7f14f80caec0 2024-04-22T07:04:59.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.883+0000 7f14ef7fe640 1 -- 172.21.15.44:0/643124558 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f0004030 con 0x7f14f80caec0 2024-04-22T07:04:59.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.883+0000 7f14fd7f4640 1 -- 172.21.15.44:0/643124558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f8154320 con 0x7f14f80caec0 2024-04-22T07:04:59.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.883+0000 7f14ef7fe640 1 -- 172.21.15.44:0/643124558 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0004400 con 0x7f14f80caec0 2024-04-22T07:04:59.883 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.883+0000 7f14fd7f4640 1 -- 172.21.15.44:0/643124558 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f81f40e0 con 0x7f14f80caec0 2024-04-22T07:04:59.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.884+0000 7f14ef7fe640 1 -- 172.21.15.44:0/643124558 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14f000c6d0 con 0x7f14f80caec0 2024-04-22T07:04:59.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.885+0000 7f14ef7fe640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f14d806c8f0 0x7f14d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.885+0000 7f14f67fc640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f14d806c8f0 0x7f14d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.885+0000 7f14ef7fe640 1 -- 172.21.15.44:0/643124558 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f14f008b960 con 0x7f14f80caec0 2024-04-22T07:04:59.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.886+0000 7f14f67fc640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f14d806c8f0 0x7f14d806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f14e0007fd0 tx=0x7f14e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:59.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.886+0000 7f15003ed740 1 Processor -- start 2024-04-22T07:04:59.884 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.886+0000 7f15003ed740 1 -- start start 2024-04-22T07:04:59.886 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f15003ed740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 0x55658c6eaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f15003ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6a2ec0 0x55658c6eb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f15003ed740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c69eca0 0x55658c6c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f15003ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55658c605b10 con 0x55658c6a2ec0 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f15003ed740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55658c5a3240 con 0x55658c6e1e40 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f15003ed740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55658c601ad0 con 0x55658c69eca0 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6a2ec0 0x55658c6eb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6a2ec0 0x55658c6eb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36926/0 (socket says 172.21.15.44:36926) 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f67fc640 1 -- 172.21.15.44:0/1352293636 learned_addr learned my addr 172.21.15.44:0/1352293636 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/1352293636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 0x55658c6eaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f77fe640 1 --2- 172.21.15.44:0/1352293636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c69eca0 0x55658c6c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f67fc640 1 -- 172.21.15.44:0/1352293636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c69eca0 msgr2=0x55658c6c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:59.887 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f67fc640 1 --2- 172.21.15.44:0/1352293636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c69eca0 0x55658c6c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f67fc640 1 -- 172.21.15.44:0/1352293636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 msgr2=0x55658c6eaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f67fc640 1 --2- 172.21.15.44:0/1352293636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 0x55658c6eaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f67fc640 1 -- 172.21.15.44:0/1352293636 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658c3182d0 con 0x55658c6a2ec0 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f77fe640 1 --2- 172.21.15.44:0/1352293636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c69eca0 0x55658c6c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.887+0000 7f14f67fc640 1 --2- 172.21.15.44:0/1352293636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6a2ec0 0x55658c6eb4a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f14f80c7070 tx=0x7f14e0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5769f5d8fd707bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f14ecff9640 1 -- 172.21.15.44:0/1352293636 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e000e500 con 0x55658c6a2ec0 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f14ecff9640 1 -- 172.21.15.44:0/1352293636 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e000e6a0 con 0x55658c6a2ec0 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f14ecff9640 1 -- 172.21.15.44:0/1352293636 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e000ea50 con 0x55658c6a2ec0 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f15003ed740 1 -- 172.21.15.44:0/1352293636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6a2ec0 msgr2=0x55658c6eb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f15003ed740 1 --2- 172.21.15.44:0/1352293636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6a2ec0 0x55658c6eb4a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f14f80c7070 tx=0x7f14e0011000 comp rx=0 tx=0).stop 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f15003ed740 1 -- 172.21.15.44:0/1352293636 shutdown_connections 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f15003ed740 1 --2- 172.21.15.44:0/1352293636 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c69eca0 0x55658c6c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f15003ed740 1 --2- 172.21.15.44:0/1352293636 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6a2ec0 0x55658c6eb4a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f15003ed740 1 --2- 172.21.15.44:0/1352293636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 0x55658c6eaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.888 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f15003ed740 1 -- 172.21.15.44:0/1352293636 >> 172.21.15.44:0/1352293636 conn(0x55658c69d990 msgr2=0x55658c6a0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:59.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f15003ed740 1 -- 172.21.15.44:0/1352293636 shutdown_connections 2024-04-22T07:04:59.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.888+0000 7f15003ed740 1 -- 172.21.15.44:0/1352293636 wait complete. 2024-04-22T07:04:59.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.889+0000 7f15003ed740 1 -- start start 2024-04-22T07:04:59.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.889+0000 7f15003ed740 1 Processor -- start 2024-04-22T07:04:59.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f15003ed740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6a2ec0 0x55658c6e7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f15003ed740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 0x55658c6e9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f15003ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6ba9a0 0x55658c6f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f15003ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55658c6a1250 con 0x55658c6ba9a0 2024-04-22T07:04:59.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f15003ed740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55658c58e190 con 0x55658c6e1e40 2024-04-22T07:04:59.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f15003ed740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55658c6a1530 con 0x55658c6a2ec0 2024-04-22T07:04:59.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f15003ed740 10 client.0 _ll_register_callbacks cb 0x55658c5898b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6ba9a0 0x55658c6f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6ba9a0 0x55658c6f5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36938/0 (socket says 172.21.15.44:36938) 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f77fe640 1 -- 172.21.15.44:0/2717118765 learned_addr learned my addr 172.21.15.44:0/2717118765 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6a2ec0 0x55658c6e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f67fc640 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 0x55658c6e9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f77fe640 1 -- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6a2ec0 msgr2=0x55658c6e7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f77fe640 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6a2ec0 0x55658c6e7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f77fe640 1 -- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 msgr2=0x55658c6e9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f77fe640 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 0x55658c6e9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f77fe640 1 -- 172.21.15.44:0/2717118765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658c3e26b0 con 0x55658c6ba9a0 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f67fc640 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 0x55658c6e9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14f77fe640 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6ba9a0 0x55658c6f5c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f14e8036040 tx=0x7f14e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892d6d15e5f6229d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:59.893 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14d5ffb640 1 -- 172.21.15.44:0/2717118765 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8009510 con 0x55658c6ba9a0 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f14d5ffb640 1 -- 172.21.15.44:0/2717118765 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f14e8034b30 con 0x55658c6ba9a0 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.890+0000 7f15003ed740 1 -- 172.21.15.44:0/2717118765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e000e6a0 con 0x55658c6ba9a0 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f14d5ffb640 1 -- 172.21.15.44:0/2717118765 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f14e8034d30 con 0x55658c6ba9a0 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f15003ed740 10 client.4887 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f15003ed740 1 -- 172.21.15.44:0/2717118765 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55658c69bf30 con 0x55658c6ba9a0 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f14d5ffb640 1 -- 172.21.15.44:0/2717118765 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e000e6a0 con 0x55658c6ba9a0 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f14d5ffb640 1 -- 172.21.15.44:0/2717118765 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f14e8048410 con 0x55658c6ba9a0 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f15003ed740 10 client.4887 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f15003ed740 10 client.4887 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f15003ed740 20 client.4887 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f15003ed740 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x55658c6f67a0 0x55658c701700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f15003ed740 4 client.4887 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:04:59.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f15003ed740 1 -- 172.21.15.44:0/2717118765 --> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55658c613530 con 0x55658c6f67a0 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.891+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x55658c6f67a0 0x55658c701700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x55658c6f67a0 0x55658c701700 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=852b4d3da70d206b server_cookie=c5df76ba8c5d1cde in_seq=0 out_seq=0 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f14d5ffb640 10 client.4887 ms_handle_connect on v2:172.21.15.70:6832/1969018412 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f14d5ffb640 1 -- 172.21.15.44:0/2717118765 <== mds.4 v2:172.21.15.70:6832/1969018412 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55658c613530 con 0x55658c6f67a0 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f14d5ffb640 10 client.4887 handle_command_reply: tid=0 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f15003ed740 1 client.4887 shutdown 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f15003ed740 1 -- 172.21.15.44:0/2717118765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6ba9a0 msgr2=0x55658c6f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f15003ed740 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6ba9a0 0x55658c6f5c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f14e8036040 tx=0x7f14e8004ab0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f15003ed740 1 -- 172.21.15.44:0/2717118765 shutdown_connections 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f15003ed740 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x55658c6f67a0 0x55658c701700 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f15003ed740 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6ba9a0 0x55658c6f5c10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f15003ed740 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6e1e40 0x55658c6e9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f15003ed740 1 --2- 172.21.15.44:0/2717118765 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6a2ec0 0x55658c6e7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f15003ed740 1 -- 172.21.15.44:0/2717118765 >> 172.21.15.44:0/2717118765 conn(0x55658c69eca0 msgr2=0x55658c6c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:59.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f14d5ffb640 0 client.4887 ms_handle_reset on v2:172.21.15.70:6832/1969018412 2024-04-22T07:04:59.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.892+0000 7f15003ed740 1 -- 172.21.15.44:0/2717118765 shutdown_connections 2024-04-22T07:04:59.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.893+0000 7f15003ed740 1 -- 172.21.15.44:0/2717118765 wait complete. 2024-04-22T07:04:59.896 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.893+0000 7f15003ed740 20 client.4887 trim_cache size 0 max 16384 2024-04-22T07:04:59.908 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.911+0000 7f15003ed740 1 Processor -- start 2024-04-22T07:04:59.909 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.911+0000 7f15003ed740 1 -- start start 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.911+0000 7f15003ed740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6a2ec0 0x55658c6ea1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f15003ed740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6ba9a0 0x55658c706730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f15003ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c706c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f15003ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f14f8151e40 con 0x55658c6e1e40 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f15003ed740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55658c6e8a90 con 0x55658c6a2ec0 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f15003ed740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55658c6e81d0 con 0x55658c6ba9a0 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c706c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c706c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36952/0 (socket says 172.21.15.44:36952) 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14f77fe640 1 -- 172.21.15.44:0/1199020079 learned_addr learned my addr 172.21.15.44:0/1199020079 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14f77fe640 1 -- 172.21.15.44:0/1199020079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6ba9a0 msgr2=0x55658c706730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:59.911 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14f77fe640 1 --2- 172.21.15.44:0/1199020079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6ba9a0 0x55658c706730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14f77fe640 1 -- 172.21.15.44:0/1199020079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6a2ec0 msgr2=0x55658c6ea1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14f77fe640 1 --2- 172.21.15.44:0/1199020079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6a2ec0 0x55658c6ea1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14f77fe640 1 -- 172.21.15.44:0/1199020079 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658c6a0090 con 0x55658c6e1e40 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14f77fe640 1 --2- 172.21.15.44:0/1199020079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c706c70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f14e8007100 tx=0x7f14e803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51073b601f7aa376 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14d7fff640 1 -- 172.21.15.44:0/1199020079 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8009510 con 0x55658c6e1e40 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14d7fff640 1 -- 172.21.15.44:0/1199020079 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e802ce50 con 0x55658c6e1e40 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.912+0000 7f14d7fff640 1 -- 172.21.15.44:0/1199020079 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8009af0 con 0x55658c6e1e40 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 -- 172.21.15.44:0/1199020079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 msgr2=0x55658c706c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 --2- 172.21.15.44:0/1199020079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c706c70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f14e8007100 tx=0x7f14e803f000 comp rx=0 tx=0).stop 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 -- 172.21.15.44:0/1199020079 shutdown_connections 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 --2- 172.21.15.44:0/1199020079 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6ba9a0 0x55658c706730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 --2- 172.21.15.44:0/1199020079 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c706c70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.912 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 --2- 172.21.15.44:0/1199020079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6a2ec0 0x55658c6ea1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 -- 172.21.15.44:0/1199020079 >> 172.21.15.44:0/1199020079 conn(0x55658c69eca0 msgr2=0x55658c696800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 -- 172.21.15.44:0/1199020079 shutdown_connections 2024-04-22T07:04:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 -- 172.21.15.44:0/1199020079 wait complete. 2024-04-22T07:04:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 -- start start 2024-04-22T07:04:59.913 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.913+0000 7f15003ed740 1 Processor -- start 2024-04-22T07:04:59.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f15003ed740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6a2ec0 0x55658c69f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f15003ed740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6ba9a0 0x55658c6e6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f15003ed740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c6e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f15003ed740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55658c6db320 con 0x55658c6e1e40 2024-04-22T07:04:59.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f15003ed740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55658c6f6620 con 0x55658c6a2ec0 2024-04-22T07:04:59.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f15003ed740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55658c6db970 con 0x55658c6ba9a0 2024-04-22T07:04:59.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f15003ed740 10 client.0 _ll_register_callbacks cb 0x7f14e8009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:04:59.916 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c6e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f14f77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c6e68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36956/0 (socket says 172.21.15.44:36956) 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f14f77fe640 1 -- 172.21.15.44:0/3291307926 learned_addr learned my addr 172.21.15.44:0/3291307926 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f14f77fe640 1 -- 172.21.15.44:0/3291307926 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6ba9a0 msgr2=0x55658c6e6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f14f77fe640 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6ba9a0 0x55658c6e6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f14f77fe640 1 -- 172.21.15.44:0/3291307926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6a2ec0 msgr2=0x55658c69f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f14f77fe640 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6a2ec0 0x55658c69f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f14f77fe640 1 -- 172.21.15.44:0/3291307926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658c6e79c0 con 0x55658c6e1e40 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f14f77fe640 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c6e68d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c090 tx=0x7f14e8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209ef1abc39d42d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.914+0000 7f14d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f14d5ffb640 1 -- 172.21.15.44:0/3291307926 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8034b30 con 0x55658c6e1e40 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f14d5ffb640 1 -- 172.21.15.44:0/3291307926 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f14e8048910 con 0x55658c6e1e40 2024-04-22T07:04:59.917 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f15003ed740 1 -- 172.21.15.44:0/3291307926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14e802ce50 con 0x55658c6e1e40 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f14d5ffb640 1 -- 172.21.15.44:0/3291307926 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f14e8009af0 con 0x55658c6e1e40 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f15003ed740 10 client.4893 fetch_fsmap learned FSMap version 17 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f15003ed740 1 -- 172.21.15.44:0/3291307926 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55658c6e7c30 con 0x55658c6e1e40 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f14d5ffb640 1 -- 172.21.15.44:0/3291307926 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e802ce50 con 0x55658c6e1e40 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f14d5ffb640 1 -- 172.21.15.44:0/3291307926 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f14e8008040 con 0x55658c6e1e40 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f15003ed740 10 client.4893 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f15003ed740 10 client.4893 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f15003ed740 20 client.4893 populate_metadata read hostname 'smithi044' 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f15003ed740 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x55658c759d60 0x55658c769490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f15003ed740 4 client.4893 mds_command: new command op to 4275 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f15003ed740 1 -- 172.21.15.44:0/3291307926 --> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] -- command(tid 0: {"prefix": "status"}) -- 0x55658c60d3a0 con 0x55658c759d60 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.915+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x55658c759d60 0x55658c769490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:04:59.918 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f14d5ffb640 10 client.4893 ms_handle_connect on v2:172.21.15.70:6832/1969018412 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f14f6ffd640 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x55658c759d60 0x55658c769490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b7b2b9c18eba800d server_cookie=bb704409a1479659 in_seq=0 out_seq=0 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f14d5ffb640 1 -- 172.21.15.44:0/3291307926 <== mds.4 v2:172.21.15.70:6832/1969018412 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55658c60d3a0 con 0x55658c759d60 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f14d5ffb640 10 client.4893 handle_command_reply: tid=0 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 client.4893 shutdown 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 -- 172.21.15.44:0/3291307926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 msgr2=0x55658c6e68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c6e68d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c090 tx=0x7f14e8009810 comp rx=0 tx=0).stop 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 -- 172.21.15.44:0/3291307926 shutdown_connections 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.70:6832/1969018412,v1:172.21.15.70:6834/1969018412] conn(0x55658c759d60 0x55658c769490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55658c6ba9a0 0x55658c6e6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f14d5ffb640 0 client.4893 ms_handle_reset on v2:172.21.15.70:6832/1969018412 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55658c6e1e40 0x55658c6e68d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 --2- 172.21.15.44:0/3291307926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55658c6a2ec0 0x55658c69f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.919 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 -- 172.21.15.44:0/3291307926 >> 172.21.15.44:0/3291307926 conn(0x55658c69eca0 msgr2=0x55658c6c5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:59.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 -- 172.21.15.44:0/3291307926 shutdown_connections 2024-04-22T07:04:59.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 1 -- 172.21.15.44:0/3291307926 wait complete. 2024-04-22T07:04:59.920 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.916+0000 7f15003ed740 20 client.4893 trim_cache size 0 max 16384 2024-04-22T07:04:59.920 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:04:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "dc21e26c-6c65-477f-a367-ffb001b78b47", 2024-04-22T07:04:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 4, 2024-04-22T07:04:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4275, 2024-04-22T07:04:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T07:04:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T07:04:59.920 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T07:04:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 19.979690550000001, 2024-04-22T07:04:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-04-22T07:04:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 25, 2024-04-22T07:04:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 23, 2024-04-22T07:04:59.921 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 52.774794716000002 2024-04-22T07:04:59.921 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 -- 172.21.15.44:0/643124558 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f14d806c8f0 msgr2=0x7f14d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f14d806c8f0 0x7f14d806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f14e0007fd0 tx=0x7f14e0009040 comp rx=0 tx=0).stop 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 -- 172.21.15.44:0/643124558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80caec0 msgr2=0x7f14f8148c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80caec0 0x7f14f8148c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f14f000e7e0 tx=0x7f14f000eca0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 -- 172.21.15.44:0/643124558 shutdown_connections 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f14d806c8f0 0x7f14d806edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f14f8155120 0x7f14f81f0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f815e8b0 0x7f14f814b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 --2- 172.21.15.44:0/643124558 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f14f80caec0 0x7f14f8148c20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 -- 172.21.15.44:0/643124558 >> 172.21.15.44:0/643124558 conn(0x7f14f80676b0 msgr2=0x7f14f8072240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 -- 172.21.15.44:0/643124558 shutdown_connections 2024-04-22T07:04:59.922 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:04:59.917+0000 7f14fd7f4640 1 -- 172.21.15.44:0/643124558 wait complete. 2024-04-22T07:04:59.925 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T07:05:00.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.029+0000 7f05797b6640 1 Processor -- start 2024-04-22T07:05:00.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.029+0000 7f05797b6640 1 -- start start 2024-04-22T07:05:00.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.030+0000 7f05797b6640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f05740c57d0 0x7f05740c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.030+0000 7f05797b6640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05740cbd70 con 0x7f0574155010 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.030+0000 7f05797b6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f057415b780 con 0x7f05740cc980 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.030+0000 7f05797b6640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f057415ba20 con 0x7f05740c57d0 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.030+0000 7f05727fc640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f05740c57d0 0x7f05740c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.030+0000 7f0572ffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f0574155010 0x7f057406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60758/0 (socket says 172.21.15.44:60758) 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.030+0000 7f05727fc640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f05740c57d0 0x7f05740c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36970/0 (socket says 172.21.15.44:36970) 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.030+0000 7f0572ffd640 1 -- 172.21.15.44:0/2997166812 learned_addr learned my addr 172.21.15.44:0/2997166812 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.031+0000 7f05727fc640 1 -- 172.21.15.44:0/2997166812 >> v1:172.21.15.70:6789/0 conn(0x7f05740cc980 legacy=0x7f05741655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.031+0000 7f05727fc640 1 -- 172.21.15.44:0/2997166812 >> v1:172.21.15.44:6789/0 conn(0x7f0574155010 legacy=0x7f057406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.031+0000 7f05727fc640 1 -- 172.21.15.44:0/2997166812 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05740cb590 con 0x7f05740c57d0 2024-04-22T07:05:00.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.031+0000 7f0571ffb640 1 -- 172.21.15.44:0/2997166812 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2890211454 0 0) 0x7f056c001110 con 0x7f0574155010 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f05727fc640 1 --2- 172.21.15.44:0/2997166812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 0x7f05740c5d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f055c0097a0 tx=0x7f055c02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2787ab1059c14e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f0571ffb640 1 -- 172.21.15.44:0/2997166812 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c02f810 con 0x7f05740c57d0 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f0571ffb640 1 -- 172.21.15.44:0/2997166812 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f055c02f9b0 con 0x7f05740c57d0 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f0571ffb640 1 -- 172.21.15.44:0/2997166812 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c02fd80 con 0x7f05740c57d0 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f05797b6640 1 -- 172.21.15.44:0/2997166812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 msgr2=0x7f05740c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f05797b6640 1 --2- 172.21.15.44:0/2997166812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 0x7f05740c5d20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f055c0097a0 tx=0x7f055c02b9f0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f05797b6640 1 -- 172.21.15.44:0/2997166812 shutdown_connections 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f05797b6640 1 --2- 172.21.15.44:0/2997166812 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 0x7f05740c5d20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f05797b6640 1 -- 172.21.15.44:0/2997166812 >> 172.21.15.44:0/2997166812 conn(0x7f0574067df0 msgr2=0x7f05740d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f05797b6640 1 -- 172.21.15.44:0/2997166812 shutdown_connections 2024-04-22T07:05:00.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.032+0000 7f05797b6640 1 -- 172.21.15.44:0/2997166812 wait complete. 2024-04-22T07:05:00.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f05797b6640 1 Processor -- start 2024-04-22T07:05:00.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f05797b6640 1 -- start start 2024-04-22T07:05:00.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f05797b6640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 0x7f0574151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f05797b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05740cc980 0x7f0574157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f05797b6640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0574155010 0x7f057415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f05797b6640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f057415e550 con 0x7f05740c57d0 2024-04-22T07:05:00.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f05797b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f057415e270 con 0x7f05740cc980 2024-04-22T07:05:00.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f05797b6640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f057415e830 con 0x7f0574155010 2024-04-22T07:05:00.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f0572ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 0x7f0574151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f0572ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 0x7f0574151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36978/0 (socket says 172.21.15.44:36978) 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.034+0000 7f0572ffd640 1 -- 172.21.15.44:0/3505206012 learned_addr learned my addr 172.21.15.44:0/3505206012 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f05727fc640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05740cc980 0x7f0574157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f05737fe640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0574155010 0x7f057415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f0572ffd640 1 -- 172.21.15.44:0/3505206012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0574155010 msgr2=0x7f057415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f0572ffd640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0574155010 0x7f057415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f0572ffd640 1 -- 172.21.15.44:0/3505206012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05740cc980 msgr2=0x7f0574157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f0572ffd640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05740cc980 0x7f0574157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f0572ffd640 1 -- 172.21.15.44:0/3505206012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05741584b0 con 0x7f05740c57d0 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f05737fe640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0574155010 0x7f057415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f0572ffd640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 0x7f0574151ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f056c00c3f0 tx=0x7f056c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f056b7fe640 1 -- 172.21.15.44:0/3505206012 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c0074e0 con 0x7f05740c57d0 2024-04-22T07:05:00.035 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f056b7fe640 1 -- 172.21.15.44:0/3505206012 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c0047e0 con 0x7f05740c57d0 2024-04-22T07:05:00.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f05797b6640 1 -- 172.21.15.44:0/3505206012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057414f8a0 con 0x7f05740c57d0 2024-04-22T07:05:00.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f056b7fe640 1 -- 172.21.15.44:0/3505206012 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c004bd0 con 0x7f05740c57d0 2024-04-22T07:05:00.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.035+0000 7f05797b6640 1 -- 172.21.15.44:0/3505206012 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05741581d0 con 0x7f05740c57d0 2024-04-22T07:05:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.037+0000 7f056b7fe640 1 -- 172.21.15.44:0/3505206012 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f056c011070 con 0x7f05740c57d0 2024-04-22T07:05:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.037+0000 7f056b7fe640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f055406c8f0 0x7f055406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.037+0000 7f056b7fe640 1 -- 172.21.15.44:0/3505206012 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f056c08b790 con 0x7f05740c57d0 2024-04-22T07:05:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.037+0000 7f05727fc640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f055406c8f0 0x7f055406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.038+0000 7f05727fc640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f055406c8f0 0x7f055406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f055c036000 tx=0x7f055c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.038+0000 7f057c3af740 1 Processor -- start 2024-04-22T07:05:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.038+0000 7f057c3af740 1 -- start start 2024-04-22T07:05:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f057c3af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f057c3af740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f47cec0 0x55747f4c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f057c3af740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f478ca0 0x55747f4a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f057c3af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55747f3dfb10 con 0x55747f4bbe40 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f057c3af740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55747f37d240 con 0x55747f47cec0 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f057c3af740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55747f3dbad0 con 0x55747f478ca0 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f0572ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f0572ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:36994/0 (socket says 172.21.15.44:36994) 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f0572ffd640 1 -- 172.21.15.44:0/4035273740 learned_addr learned my addr 172.21.15.44:0/4035273740 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f05737fe640 1 --2- 172.21.15.44:0/4035273740 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f478ca0 0x55747f4a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f0572ffd640 1 -- 172.21.15.44:0/4035273740 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f478ca0 msgr2=0x55747f4a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f05727fc640 1 --2- 172.21.15.44:0/4035273740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f47cec0 0x55747f4c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f0572ffd640 1 --2- 172.21.15.44:0/4035273740 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f478ca0 0x55747f4a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f0572ffd640 1 -- 172.21.15.44:0/4035273740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f47cec0 msgr2=0x55747f4c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f0572ffd640 1 --2- 172.21.15.44:0/4035273740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f47cec0 0x55747f4c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f0572ffd640 1 -- 172.21.15.44:0/4035273740 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747f0f22d0 con 0x55747f4bbe40 2024-04-22T07:05:00.044 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.039+0000 7f0572ffd640 1 --2- 172.21.15.44:0/4035273740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c4f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f056c089be0 tx=0x7f056c058e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d920b23b7eab66e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f0568ff9640 1 -- 172.21.15.44:0/4035273740 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c059440 con 0x55747f4bbe40 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f0568ff9640 1 -- 172.21.15.44:0/4035273740 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c06de50 con 0x55747f4bbe40 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f0568ff9640 1 -- 172.21.15.44:0/4035273740 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c05d7f0 con 0x55747f4bbe40 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f057c3af740 1 -- 172.21.15.44:0/4035273740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 msgr2=0x55747f4c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f057c3af740 1 --2- 172.21.15.44:0/4035273740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c4f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f056c089be0 tx=0x7f056c058e60 comp rx=0 tx=0).stop 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f057c3af740 1 -- 172.21.15.44:0/4035273740 shutdown_connections 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f057c3af740 1 --2- 172.21.15.44:0/4035273740 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f478ca0 0x55747f4a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f057c3af740 1 --2- 172.21.15.44:0/4035273740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f47cec0 0x55747f4c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f057c3af740 1 --2- 172.21.15.44:0/4035273740 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c4f60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f057c3af740 1 -- 172.21.15.44:0/4035273740 >> 172.21.15.44:0/4035273740 conn(0x55747f477990 msgr2=0x55747f47a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f057c3af740 1 -- 172.21.15.44:0/4035273740 shutdown_connections 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.040+0000 7f057c3af740 1 -- 172.21.15.44:0/4035273740 wait complete. 2024-04-22T07:05:00.045 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.041+0000 7f057c3af740 1 -- start start 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.041+0000 7f057c3af740 1 Processor -- start 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f057c3af740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f475140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f057c3af740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bbe40 0x55747f4becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f057c3af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f057c3af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55747f47b250 con 0x55747f4bdfb0 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f057c3af740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55747f368190 con 0x55747f4bbe40 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f057c3af740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55747f47b530 con 0x55747f47cec0 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f057c3af740 10 client.0 _ll_register_callbacks cb 0x55747f3638b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f05737fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f05737fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4cfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37006/0 (socket says 172.21.15.44:37006) 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f05737fe640 1 -- 172.21.15.44:0/265808894 learned_addr learned my addr 172.21.15.44:0/265808894 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f0572ffd640 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f475140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.046 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f05727fc640 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bbe40 0x55747f4becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f05737fe640 1 -- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 msgr2=0x55747f475140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f05737fe640 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f475140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f05737fe640 1 -- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bbe40 msgr2=0x55747f4becd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f05737fe640 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bbe40 0x55747f4becd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f05737fe640 1 -- 172.21.15.44:0/265808894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747f1bc6b0 con 0x55747f4bdfb0 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f0572ffd640 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f475140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f05737fe640 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4cfcc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f056400e4d0 tx=0x7f056400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793fa2a49ca13750 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.042+0000 7f0551ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f0551ffb640 1 -- 172.21.15.44:0/265808894 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564003d80 con 0x55747f4bdfb0 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f0551ffb640 1 -- 172.21.15.44:0/265808894 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f05640050d0 con 0x55747f4bdfb0 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f057c3af740 1 -- 172.21.15.44:0/265808894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f056c06de50 con 0x55747f4bdfb0 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f0551ffb640 1 -- 172.21.15.44:0/265808894 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f056401f050 con 0x55747f4bdfb0 2024-04-22T07:05:00.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f057c3af740 10 client.4908 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f057c3af740 1 -- 172.21.15.44:0/265808894 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55747f49dac0 con 0x55747f4bdfb0 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f0551ffb640 1 -- 172.21.15.44:0/265808894 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c06de50 con 0x55747f4bdfb0 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f0551ffb640 1 -- 172.21.15.44:0/265808894 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f0564022020 con 0x55747f4bdfb0 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f057c3af740 10 client.4908 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f057c3af740 10 client.4908 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f057c3af740 20 client.4908 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f057c3af740 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x55747f4d0850 0x55747f4db840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f057c3af740 4 client.4908 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f057c3af740 1 -- 172.21.15.44:0/265808894 --> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55747f3ed530 con 0x55747f4d0850 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.043+0000 7f0572ffd640 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x55747f4d0850 0x55747f4db840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.044+0000 7f0572ffd640 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x55747f4d0850 0x55747f4db840 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d60cd4baf0949b1b server_cookie=34ddf7e85e065c3c in_seq=0 out_seq=0 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.044+0000 7f0551ffb640 10 client.4908 ms_handle_connect on v2:172.21.15.70:6833/4058009549 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.044+0000 7f0551ffb640 1 -- 172.21.15.44:0/265808894 <== mds.3 v2:172.21.15.70:6833/4058009549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55747f3ed530 con 0x55747f4d0850 2024-04-22T07:05:00.048 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.044+0000 7f0551ffb640 10 client.4908 handle_command_reply: tid=0 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.044+0000 7f057c3af740 1 client.4908 shutdown 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 1 -- 172.21.15.44:0/265808894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 msgr2=0x55747f4cfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4cfcc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f056400e4d0 tx=0x7f056400e990 comp rx=0 tx=0).stop 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 1 -- 172.21.15.44:0/265808894 shutdown_connections 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x55747f4d0850 0x55747f4db840 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4cfcc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bbe40 0x55747f4becd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f0551ffb640 0 client.4908 ms_handle_reset on v2:172.21.15.70:6833/4058009549 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 1 --2- 172.21.15.44:0/265808894 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f475140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 1 -- 172.21.15.44:0/265808894 >> 172.21.15.44:0/265808894 conn(0x55747f478ca0 msgr2=0x55747f4babd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 1 -- 172.21.15.44:0/265808894 shutdown_connections 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 1 -- 172.21.15.44:0/265808894 wait complete. 2024-04-22T07:05:00.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.045+0000 7f057c3af740 20 client.4908 trim_cache size 0 max 16384 2024-04-22T07:05:00.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.063+0000 7f057c3af740 1 Processor -- start 2024-04-22T07:05:00.061 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.063+0000 7f057c3af740 1 -- start start 2024-04-22T07:05:00.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f057c3af740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f4b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f057c3af740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bbe40 0x55747f4c4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f057c3af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f057c3af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f05740cf490 con 0x55747f4bdfb0 2024-04-22T07:05:00.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f057c3af740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55747f4c4810 con 0x55747f4bbe40 2024-04-22T07:05:00.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f057c3af740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55747f4b9940 con 0x55747f47cec0 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f05737fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f05737fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37018/0 (socket says 172.21.15.44:37018) 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f05737fe640 1 -- 172.21.15.44:0/3086196461 learned_addr learned my addr 172.21.15.44:0/3086196461 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f0572ffd640 1 --2- 172.21.15.44:0/3086196461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f4b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f05727fc640 1 --2- 172.21.15.44:0/3086196461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bbe40 0x55747f4c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f05737fe640 1 -- 172.21.15.44:0/3086196461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 msgr2=0x55747f4b9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f05737fe640 1 --2- 172.21.15.44:0/3086196461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f4b9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f05737fe640 1 -- 172.21.15.44:0/3086196461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bbe40 msgr2=0x55747f4c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f05737fe640 1 --2- 172.21.15.44:0/3086196461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bbe40 0x55747f4c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f05737fe640 1 -- 172.21.15.44:0/3086196461 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747f4bcf50 con 0x55747f4bdfb0 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f0572ffd640 1 --2- 172.21.15.44:0/3086196461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f4b9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:00.064 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.064+0000 7f05737fe640 1 --2- 172.21.15.44:0/3086196461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4c4ed0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0564004fd0 tx=0x7f056400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b31fa358fb4ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f0553fff640 1 -- 172.21.15.44:0/3086196461 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564003b80 con 0x55747f4bdfb0 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f0553fff640 1 -- 172.21.15.44:0/3086196461 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0564003d40 con 0x55747f4bdfb0 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f0553fff640 1 -- 172.21.15.44:0/3086196461 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0564006d90 con 0x55747f4bdfb0 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f057c3af740 1 -- 172.21.15.44:0/3086196461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 msgr2=0x55747f4c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f057c3af740 1 --2- 172.21.15.44:0/3086196461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4c4ed0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0564004fd0 tx=0x7f056400c000 comp rx=0 tx=0).stop 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f057c3af740 1 -- 172.21.15.44:0/3086196461 shutdown_connections 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f057c3af740 1 --2- 172.21.15.44:0/3086196461 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bdfb0 0x55747f4c4ed0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f057c3af740 1 --2- 172.21.15.44:0/3086196461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bbe40 0x55747f4c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f057c3af740 1 --2- 172.21.15.44:0/3086196461 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f4b9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f057c3af740 1 -- 172.21.15.44:0/3086196461 >> 172.21.15.44:0/3086196461 conn(0x55747f478ca0 msgr2=0x55747f4cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f057c3af740 1 -- 172.21.15.44:0/3086196461 shutdown_connections 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.065+0000 7f057c3af740 1 -- 172.21.15.44:0/3086196461 wait complete. 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f057c3af740 1 -- start start 2024-04-22T07:05:00.065 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f057c3af740 1 Processor -- start 2024-04-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f057c3af740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f4ba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f057c3af740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f057c3af740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bdfb0 0x55747f4c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f057c3af740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55747f492420 con 0x55747f4bbe40 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f057c3af740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55747f4d06d0 con 0x55747f4bdfb0 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f057c3af740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55747f492a70 con 0x55747f47cec0 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f057c3af740 10 client.0 _ll_register_callbacks cb 0x7f0564001540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f05727fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f05727fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37024/0 (socket says 172.21.15.44:37024) 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.066+0000 7f05727fc640 1 -- 172.21.15.44:0/2060333947 learned_addr learned my addr 172.21.15.44:0/2060333947 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f05737fe640 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bdfb0 0x55747f4c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f0572ffd640 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f4ba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f05727fc640 1 -- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 msgr2=0x55747f4ba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f05727fc640 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f4ba0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.070 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f05727fc640 1 -- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bdfb0 msgr2=0x55747f4c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f05727fc640 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bdfb0 0x55747f4c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f05727fc640 1 -- 172.21.15.44:0/2060333947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747f4c4f60 con 0x55747f4bbe40 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f05727fc640 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c3190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f055c02ba30 tx=0x7f055c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b7ea9bf23a0515 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f0551ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f0551ffb640 1 -- 172.21.15.44:0/2060333947 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c056020 con 0x55747f4bbe40 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f0551ffb640 1 -- 172.21.15.44:0/2060333947 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f055c02fa20 con 0x55747f4bbe40 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f057c3af740 1 -- 172.21.15.44:0/2060333947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0564003d40 con 0x55747f4bbe40 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f0551ffb640 1 -- 172.21.15.44:0/2060333947 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f055c060050 con 0x55747f4bbe40 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f057c3af740 10 client.4914 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.067+0000 7f057c3af740 1 -- 172.21.15.44:0/2060333947 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55747f4bb1a0 con 0x55747f4bbe40 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f0551ffb640 1 -- 172.21.15.44:0/2060333947 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0564003d40 con 0x55747f4bbe40 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f0551ffb640 1 -- 172.21.15.44:0/2060333947 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f055c063020 con 0x55747f4bbe40 2024-04-22T07:05:00.071 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f057c3af740 10 client.4914 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f057c3af740 10 client.4914 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f057c3af740 20 client.4914 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f057c3af740 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x55747f48f720 0x55747f543810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f057c3af740 4 client.4914 mds_command: new command op to 4278 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f057c3af740 1 -- 172.21.15.44:0/2060333947 --> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] -- command(tid 0: {"prefix": "status"}) -- 0x55747f3e73a0 con 0x55747f48f720 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f0572ffd640 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x55747f48f720 0x55747f543810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f0572ffd640 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x55747f48f720 0x55747f543810 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=128ffb000b0fe2b1 server_cookie=e294dd922c6d7cc1 in_seq=0 out_seq=0 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f0551ffb640 10 client.4914 ms_handle_connect on v2:172.21.15.70:6833/4058009549 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f0551ffb640 1 -- 172.21.15.44:0/2060333947 <== mds.3 v2:172.21.15.70:6833/4058009549 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55747f3e73a0 con 0x55747f48f720 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f0551ffb640 10 client.4914 handle_command_reply: tid=0 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.068+0000 7f057c3af740 1 client.4914 shutdown 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 1 -- 172.21.15.44:0/2060333947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 msgr2=0x55747f4c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c3190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f055c02ba30 tx=0x7f055c004730 comp rx=0 tx=0).stop 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 1 -- 172.21.15.44:0/2060333947 shutdown_connections 2024-04-22T07:05:00.072 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:6833/4058009549,v1:172.21.15.70:6835/4058009549] conn(0x55747f48f720 0x55747f543810 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55747f4bdfb0 0x55747f4c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55747f4bbe40 0x55747f4c3190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f0551ffb640 0 client.4914 ms_handle_reset on v2:172.21.15.70:6833/4058009549 2024-04-22T07:05:00.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 1 --2- 172.21.15.44:0/2060333947 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55747f47cec0 0x55747f4ba0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 1 -- 172.21.15.44:0/2060333947 >> 172.21.15.44:0/2060333947 conn(0x55747f478ca0 msgr2=0x55747f4b9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 1 -- 172.21.15.44:0/2060333947 shutdown_connections 2024-04-22T07:05:00.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 1 -- 172.21.15.44:0/2060333947 wait complete. 2024-04-22T07:05:00.073 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f057c3af740 20 client.4914 trim_cache size 0 max 16384 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "dc21e26c-6c65-477f-a367-ffb001b78b47", 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 3, 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4278, 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 25.203152698, 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 25, 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 23, 2024-04-22T07:05:00.074 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 52.927163329999999 2024-04-22T07:05:00.075 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:05:00.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 -- 172.21.15.44:0/3505206012 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f055406c8f0 msgr2=0x7f055406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.075 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f055406c8f0 0x7f055406edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f055c036000 tx=0x7f055c008030 comp rx=0 tx=0).stop 2024-04-22T07:05:00.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 -- 172.21.15.44:0/3505206012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 msgr2=0x7f0574151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 0x7f0574151ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f056c00c3f0 tx=0x7f056c00c8b0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 -- 172.21.15.44:0/3505206012 shutdown_connections 2024-04-22T07:05:00.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f055406c8f0 0x7f055406edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0574155010 0x7f057415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05740cc980 0x7f0574157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 --2- 172.21.15.44:0/3505206012 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f05740c57d0 0x7f0574151ea0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 -- 172.21.15.44:0/3505206012 >> 172.21.15.44:0/3505206012 conn(0x7f0574067df0 msgr2=0x7f0574152830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 -- 172.21.15.44:0/3505206012 shutdown_connections 2024-04-22T07:05:00.076 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.069+0000 7f05797b6640 1 -- 172.21.15.44:0/3505206012 wait complete. 2024-04-22T07:05:00.077 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-22T07:05:00.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.181+0000 7fbaa954b640 1 Processor -- start 2024-04-22T07:05:00.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.181+0000 7fbaa954b640 1 -- start start 2024-04-22T07:05:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.182+0000 7fbaa954b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbaa40ca7a0 0x7fbaa40cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.182+0000 7fbaa954b640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa4157c00 con 0x7fbaa4151df0 2024-04-22T07:05:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.182+0000 7fbaa954b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa40c3830 con 0x7fbaa40cccf0 2024-04-22T07:05:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.182+0000 7fbaa954b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa40cbeb0 con 0x7fbaa40ca7a0 2024-04-22T07:05:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.182+0000 7fbaa2ffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fbaa4151df0 0x7fbaa40c9f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60764/0 (socket says 172.21.15.44:60764) 2024-04-22T07:05:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.182+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/999318389 learned_addr learned my addr 172.21.15.44:0/999318389 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:05:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.182+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160214877 0 0) 0x7fbaa4157c00 con 0x7fbaa4151df0 2024-04-22T07:05:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.182+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba90003520 con 0x7fbaa4151df0 2024-04-22T07:05:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.182+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/999318389 >> v2:172.21.15.70:3300/0 conn(0x7fbaa40ca7a0 0x7fbaa40cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.182+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1580282498 0 0) 0x7fba90003520 con 0x7fbaa4151df0 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 >> v2:172.21.15.70:3300/0 conn(0x7fbaa40ca7a0 msgr2=0x7fbaa40cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 --2- 172.21.15.44:0/999318389 >> v2:172.21.15.70:3300/0 conn(0x7fbaa40ca7a0 0x7fbaa40cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 >> v1:172.21.15.70:6789/0 conn(0x7fbaa40cccf0 legacy=0x7fbaa40c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa40cb5c0 con 0x7fbaa4151df0 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3718881788 0 0) 0x7fba8c001670 con 0x7fbaa4151df0 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 >> v1:172.21.15.44:6789/0 conn(0x7fbaa4151df0 legacy=0x7fbaa40c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba900039b0 0x7fba90003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90004800 0x7fba90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba900051c0 0x7fba9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40cbeb0 con 0x7fba900051c0 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fba9000c070 con 0x7fba90004800 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fba9000c270 con 0x7fba900039b0 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba900051c0 0x7fba9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90004800 0x7fba90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba900039b0 0x7fba90003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa37fe640 1 -- 172.21.15.44:0/999318389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba900039b0 msgr2=0x7fba90003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba900039b0 0x7fba90003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa37fe640 1 -- 172.21.15.44:0/999318389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90004800 msgr2=0x7fba90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90004800 0x7fba90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa37fe640 1 -- 172.21.15.44:0/999318389 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba90003520 con 0x7fba900051c0 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.183+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90004800 0x7fba90004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.184+0000 7fbaa37fe640 1 -- 172.21.15.44:0/999318389 reap_dead start 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.184+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba900051c0 0x7fba9000b9c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009fc0 tx=0x7fba90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7d3093380def26 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.184+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94001370 con 0x7fba900051c0 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.184+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c02cca0 con 0x7fba900051c0 2024-04-22T07:05:00.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.184+0000 7fbaa1ffb640 1 -- 172.21.15.44:0/999318389 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c009c10 con 0x7fba900051c0 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.185+0000 7fbaa954b640 1 -- 172.21.15.44:0/999318389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba900051c0 msgr2=0x7fba9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.185+0000 7fbaa954b640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba900051c0 0x7fba9000b9c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009fc0 tx=0x7fba90005830 comp rx=0 tx=0).stop 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.185+0000 7fbaa954b640 1 -- 172.21.15.44:0/999318389 shutdown_connections 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.185+0000 7fbaa954b640 1 --2- 172.21.15.44:0/999318389 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fba900051c0 0x7fba9000b9c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.185+0000 7fbaa954b640 1 -- 172.21.15.44:0/999318389 >> 172.21.15.44:0/999318389 conn(0x7fbaa406b970 msgr2=0x7fbaa406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.185+0000 7fbaa954b640 1 -- 172.21.15.44:0/999318389 shutdown_connections 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.185+0000 7fbaa954b640 1 -- 172.21.15.44:0/999318389 wait complete. 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.186+0000 7fbaa954b640 1 Processor -- start 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.186+0000 7fbaa954b640 1 -- start start 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa954b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba900051c0 0x7fbaa414f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa954b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaa40c2e10 0x7fbaa41ee870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa954b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa41464c0 0x7fbaa4142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa954b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40cbb40 con 0x7fbaa41464c0 2024-04-22T07:05:00.188 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa954b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40cb860 con 0x7fbaa40c2e10 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa954b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa40c3db0 con 0x7fba900051c0 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa41464c0 0x7fbaa4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa41464c0 0x7fbaa4142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37046/0 (socket says 172.21.15.44:37046) 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa37fe640 1 -- 172.21.15.44:0/3832304637 learned_addr learned my addr 172.21.15.44:0/3832304637 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba900051c0 0x7fbaa414f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaa40c2e10 0x7fbaa41ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa37fe640 1 -- 172.21.15.44:0/3832304637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba900051c0 msgr2=0x7fbaa414f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba900051c0 0x7fbaa414f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa37fe640 1 -- 172.21.15.44:0/3832304637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaa40c2e10 msgr2=0x7fbaa41ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaa40c2e10 0x7fbaa41ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa37fe640 1 -- 172.21.15.44:0/3832304637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa40c6a90 con 0x7fbaa41464c0 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaa40c2e10 0x7fbaa41ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:00.189 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa41464c0 0x7fbaa4142680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00e200 tx=0x7fba9c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fba9b7fe640 1 -- 172.21.15.44:0/3832304637 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c003990 con 0x7fbaa41464c0 2024-04-22T07:05:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fba9b7fe640 1 -- 172.21.15.44:0/3832304637 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c044070 con 0x7fbaa41464c0 2024-04-22T07:05:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fbaa954b640 1 -- 172.21.15.44:0/3832304637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa40c6d30 con 0x7fbaa41464c0 2024-04-22T07:05:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.187+0000 7fba9b7fe640 1 -- 172.21.15.44:0/3832304637 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c003d90 con 0x7fbaa41464c0 2024-04-22T07:05:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.188+0000 7fbaa954b640 1 -- 172.21.15.44:0/3832304637 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa40c04f0 con 0x7fbaa41464c0 2024-04-22T07:05:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.189+0000 7fba9b7fe640 1 -- 172.21.15.44:0/3832304637 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba9c005d10 con 0x7fbaa41464c0 2024-04-22T07:05:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.190+0000 7fba9b7fe640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fba8406c8f0 0x7fba8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.190+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fba8406c8f0 0x7fba8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.190+0000 7fba9b7fe640 1 -- 172.21.15.44:0/3832304637 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7fba9c0ae320 con 0x7fbaa41464c0 2024-04-22T07:05:00.190 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.190+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fba8406c8f0 0x7fba8406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba8c001eb0 tx=0x7fba8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.191+0000 7fbaac211740 1 Processor -- start 2024-04-22T07:05:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.191+0000 7fbaac211740 1 -- start start 2024-04-22T07:05:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.191+0000 7fbaac211740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385e9e40 0x556d385f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.191+0000 7fbaac211740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385aaec0 0x556d385f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.191+0000 7fbaac211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385a6ca0 0x556d385ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.191+0000 7fbaac211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556d3850db10 con 0x556d385a6ca0 2024-04-22T07:05:00.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.191+0000 7fbaac211740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x556d384ab240 con 0x556d385aaec0 2024-04-22T07:05:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.191+0000 7fbaac211740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x556d38509ad0 con 0x556d385e9e40 2024-04-22T07:05:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385a6ca0 0x556d385ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa37fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385a6ca0 0x556d385ceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37058/0 (socket says 172.21.15.44:37058) 2024-04-22T07:05:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa37fe640 1 -- 172.21.15.44:0/1239551222 learned_addr learned my addr 172.21.15.44:0/1239551222 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/1239551222 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385e9e40 0x556d385f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/1239551222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385aaec0 0x556d385f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa37fe640 1 -- 172.21.15.44:0/1239551222 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385e9e40 msgr2=0x556d385f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/1239551222 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385e9e40 0x556d385f2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa37fe640 1 -- 172.21.15.44:0/1239551222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385aaec0 msgr2=0x556d385f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/1239551222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385aaec0 0x556d385f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa37fe640 1 -- 172.21.15.44:0/1239551222 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d382202d0 con 0x556d385a6ca0 2024-04-22T07:05:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/1239551222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385aaec0 0x556d385f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/1239551222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385a6ca0 0x556d385ceee0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0095c0 tx=0x7fba9c07b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf60fdee94064e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fba98ff9640 1 -- 172.21.15.44:0/1239551222 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0802f0 con 0x556d385a6ca0 2024-04-22T07:05:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fba98ff9640 1 -- 172.21.15.44:0/1239551222 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c02ce50 con 0x556d385a6ca0 2024-04-22T07:05:00.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.192+0000 7fba98ff9640 1 -- 172.21.15.44:0/1239551222 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c084bb0 con 0x556d385a6ca0 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.193+0000 7fbaac211740 1 -- 172.21.15.44:0/1239551222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385a6ca0 msgr2=0x556d385ceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.193+0000 7fbaac211740 1 --2- 172.21.15.44:0/1239551222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385a6ca0 0x556d385ceee0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0095c0 tx=0x7fba9c07b960 comp rx=0 tx=0).stop 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.193+0000 7fbaac211740 1 -- 172.21.15.44:0/1239551222 shutdown_connections 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.193+0000 7fbaac211740 1 --2- 172.21.15.44:0/1239551222 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385a6ca0 0x556d385ceee0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.193+0000 7fbaac211740 1 --2- 172.21.15.44:0/1239551222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385aaec0 0x556d385f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.193+0000 7fbaac211740 1 --2- 172.21.15.44:0/1239551222 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385e9e40 0x556d385f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.193+0000 7fbaac211740 1 -- 172.21.15.44:0/1239551222 >> 172.21.15.44:0/1239551222 conn(0x556d385a5990 msgr2=0x556d385a8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.193+0000 7fbaac211740 1 -- 172.21.15.44:0/1239551222 shutdown_connections 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.193+0000 7fbaac211740 1 -- 172.21.15.44:0/1239551222 wait complete. 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.194+0000 7fbaac211740 1 -- start start 2024-04-22T07:05:00.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.194+0000 7fbaac211740 1 Processor -- start 2024-04-22T07:05:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.194+0000 7fbaac211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385eb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.194+0000 7fbaac211740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385e9e40 0x556d385f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.194+0000 7fbaac211740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385ebb20 0x556d385a2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.194+0000 7fbaac211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556d384ab240 con 0x556d385aaec0 2024-04-22T07:05:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.194+0000 7fbaac211740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x556d38496190 con 0x556d385ebb20 2024-04-22T07:05:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.194+0000 7fbaac211740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x556d3850db10 con 0x556d385e9e40 2024-04-22T07:05:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385eb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaac211740 10 client.0 _ll_register_callbacks cb 0x556d384918b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:00.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385eb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37068/0 (socket says 172.21.15.44:37068) 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/2453451385 learned_addr learned my addr 172.21.15.44:0/2453451385 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385e9e40 0x556d385f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa37fe640 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385ebb20 0x556d385a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/2453451385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385e9e40 msgr2=0x556d385f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385e9e40 0x556d385f1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/2453451385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385ebb20 msgr2=0x556d385a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385ebb20 0x556d385a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/2453451385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d382ea6b0 con 0x556d385aaec0 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385eb280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0157c0 tx=0x7fba8c0078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69edce7f4f94b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fba81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fba81ffb640 1 -- 172.21.15.44:0/2453451385 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c027030 con 0x556d385aaec0 2024-04-22T07:05:00.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fba81ffb640 1 -- 172.21.15.44:0/2453451385 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7fba8c010040 con 0x556d385aaec0 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaac211740 1 -- 172.21.15.44:0/2453451385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba9c02ce50 con 0x556d385aaec0 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fba81ffb640 1 -- 172.21.15.44:0/2453451385 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fba8c014950 con 0x556d385aaec0 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaac211740 10 client.4926 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.195+0000 7fbaac211740 1 -- 172.21.15.44:0/2453451385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x556d385a47a0 con 0x556d385aaec0 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fba81ffb640 1 -- 172.21.15.44:0/2453451385 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c02ce50 con 0x556d385aaec0 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fba81ffb640 1 -- 172.21.15.44:0/2453451385 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fba8c014af0 con 0x556d385aaec0 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fbaac211740 10 client.4926 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fbaac211740 10 client.4926 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fbaac211740 20 client.4926 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fbaac211740 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x556d386072e0 0x556d386096d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fbaac211740 4 client.4926 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fbaac211740 1 -- 172.21.15.44:0/2453451385 --> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d3851b530 con 0x556d386072e0 2024-04-22T07:05:00.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x556d386072e0 0x556d386096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x556d386072e0 0x556d386096d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9aaaebae2c3f4e0 server_cookie=183d962305a1a9a2 in_seq=0 out_seq=0 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.196+0000 7fba81ffb640 10 client.4926 ms_handle_connect on v2:172.21.15.44:6836/1520421043 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fba81ffb640 1 -- 172.21.15.44:0/2453451385 <== mds.2 v2:172.21.15.44:6836/1520421043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d3851b530 con 0x556d386072e0 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fba81ffb640 10 client.4926 handle_command_reply: tid=0 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 client.4926 shutdown 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 -- 172.21.15.44:0/2453451385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 msgr2=0x556d385eb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385eb280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0157c0 tx=0x7fba8c0078f0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 -- 172.21.15.44:0/2453451385 shutdown_connections 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x556d386072e0 0x556d386096d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385e9e40 0x556d385f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385ebb20 0x556d385a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fba81ffb640 0 client.4926 ms_handle_reset on v2:172.21.15.44:6836/1520421043 2024-04-22T07:05:00.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 --2- 172.21.15.44:0/2453451385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385eb280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 -- 172.21.15.44:0/2453451385 >> 172.21.15.44:0/2453451385 conn(0x556d385a6ca0 msgr2=0x556d385f2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 -- 172.21.15.44:0/2453451385 shutdown_connections 2024-04-22T07:05:00.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 1 -- 172.21.15.44:0/2453451385 wait complete. 2024-04-22T07:05:00.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.197+0000 7fbaac211740 20 client.4926 trim_cache size 0 max 16384 2024-04-22T07:05:00.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.215+0000 7fbaac211740 1 Processor -- start 2024-04-22T07:05:00.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.215+0000 7fbaac211740 1 -- start start 2024-04-22T07:05:00.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaac211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaac211740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385e9e40 0x556d385cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.214 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaac211740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385ebb20 0x556d385ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaac211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41fb850 con 0x556d385aaec0 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaac211740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa40cc6f0 con 0x556d385e9e40 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaac211740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x556d385a39e0 con 0x556d385ebb20 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37072/0 (socket says 172.21.15.44:37072) 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/3643683385 learned_addr learned my addr 172.21.15.44:0/3643683385 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/3643683385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385ebb20 msgr2=0x556d385ef3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/3643683385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385ebb20 0x556d385ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/3643683385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385e9e40 msgr2=0x556d385cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/3643683385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385e9e40 0x556d385cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/3643683385 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d385a24e0 con 0x556d385aaec0 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.216+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/3643683385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cc160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fba8c001970 tx=0x7fba8c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9c5fcce83b6366 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fba83fff640 1 -- 172.21.15.44:0/3643683385 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c027070 con 0x556d385aaec0 2024-04-22T07:05:00.215 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fba83fff640 1 -- 172.21.15.44:0/3643683385 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8c028040 con 0x556d385aaec0 2024-04-22T07:05:00.216 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fba83fff640 1 -- 172.21.15.44:0/3643683385 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c022430 con 0x556d385aaec0 2024-04-22T07:05:00.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fbaac211740 1 -- 172.21.15.44:0/3643683385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 msgr2=0x556d385cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fbaac211740 1 --2- 172.21.15.44:0/3643683385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cc160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fba8c001970 tx=0x7fba8c01b000 comp rx=0 tx=0).stop 2024-04-22T07:05:00.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fbaac211740 1 -- 172.21.15.44:0/3643683385 shutdown_connections 2024-04-22T07:05:00.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fbaac211740 1 --2- 172.21.15.44:0/3643683385 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385ebb20 0x556d385ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.219 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fbaac211740 1 --2- 172.21.15.44:0/3643683385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385e9e40 0x556d385cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fbaac211740 1 --2- 172.21.15.44:0/3643683385 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cc160 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fbaac211740 1 -- 172.21.15.44:0/3643683385 >> 172.21.15.44:0/3643683385 conn(0x556d385a6ca0 msgr2=0x556d385cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fbaac211740 1 -- 172.21.15.44:0/3643683385 shutdown_connections 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.217+0000 7fbaac211740 1 -- 172.21.15.44:0/3643683385 wait complete. 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.218+0000 7fbaac211740 1 -- start start 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.218+0000 7fbaac211740 1 Processor -- start 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.218+0000 7fbaac211740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.218+0000 7fbaac211740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385e9e40 0x556d385e6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.218+0000 7fbaac211740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385ebb20 0x556d3865df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.218+0000 7fbaac211740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556d385ccc90 con 0x556d385aaec0 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.218+0000 7fbaac211740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x556d385f24c0 con 0x556d385e9e40 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.218+0000 7fbaac211740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x556d385fe080 con 0x556d385ebb20 2024-04-22T07:05:00.220 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaac211740 10 client.0 _ll_register_callbacks cb 0x7fba8c007720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37074/0 (socket says 172.21.15.44:37074) 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/3791456411 learned_addr learned my addr 172.21.15.44:0/3791456411 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385e9e40 0x556d385e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/3791456411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385ebb20 msgr2=0x556d3865df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385ebb20 0x556d3865df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/3791456411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385e9e40 msgr2=0x556d385e6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385e9e40 0x556d385e6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaa2ffd640 1 -- 172.21.15.44:0/3791456411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d385e6da0 con 0x556d385aaec0 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaa2ffd640 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cb650 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba8c024040 tx=0x7fba8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688f9780b40a4024 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fba81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fba81ffb640 1 -- 172.21.15.44:0/3791456411 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8c043020 con 0x556d385aaec0 2024-04-22T07:05:00.221 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fba81ffb640 1 -- 172.21.15.44:0/3791456411 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7fba8c03f460 con 0x556d385aaec0 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaac211740 1 -- 172.21.15.44:0/3791456411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba8c028040 con 0x556d385aaec0 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fba81ffb640 1 -- 172.21.15.44:0/3791456411 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fba8c04d050 con 0x556d385aaec0 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaac211740 10 client.4932 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fbaac211740 1 -- 172.21.15.44:0/3791456411 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x556d385f1170 con 0x556d385aaec0 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.219+0000 7fba81ffb640 1 -- 172.21.15.44:0/3791456411 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8c028040 con 0x556d385aaec0 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fba81ffb640 1 -- 172.21.15.44:0/3791456411 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7fba8c050020 con 0x556d385aaec0 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 10 client.4932 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 10 client.4932 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 20 client.4932 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x556d385f3300 0x556d386709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 4 client.4932 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 -- 172.21.15.44:0/3791456411 --> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] -- command(tid 0: {"prefix": "status"}) -- 0x556d385e8420 con 0x556d385f3300 2024-04-22T07:05:00.222 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x556d385f3300 0x556d386709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fba81ffb640 10 client.4932 ms_handle_connect on v2:172.21.15.44:6836/1520421043 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaa27fc640 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x556d385f3300 0x556d386709c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b4247c92be6f57d server_cookie=22c59779a9a19c96 in_seq=0 out_seq=0 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fba81ffb640 1 -- 172.21.15.44:0/3791456411 <== mds.2 v2:172.21.15.44:6836/1520421043 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x556d385e8420 con 0x556d385f3300 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fba81ffb640 10 client.4932 handle_command_reply: tid=0 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 client.4932 shutdown 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 -- 172.21.15.44:0/3791456411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 msgr2=0x556d385cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cb650 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba8c024040 tx=0x7fba8c017000 comp rx=0 tx=0).stop 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 -- 172.21.15.44:0/3791456411 shutdown_connections 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.44:6836/1520421043,v1:172.21.15.44:6838/1520421043] conn(0x556d385f3300 0x556d386709c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d385ebb20 0x556d3865df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d385e9e40 0x556d385e6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 --2- 172.21.15.44:0/3791456411 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d385aaec0 0x556d385cb650 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fba81ffb640 0 client.4932 ms_handle_reset on v2:172.21.15.44:6836/1520421043 2024-04-22T07:05:00.223 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 -- 172.21.15.44:0/3791456411 >> 172.21.15.44:0/3791456411 conn(0x556d385a6ca0 msgr2=0x556d385cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 -- 172.21.15.44:0/3791456411 shutdown_connections 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.220+0000 7fbaac211740 1 -- 172.21.15.44:0/3791456411 wait complete. 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaac211740 20 client.4932 trim_cache size 0 max 16384 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "dc21e26c-6c65-477f-a367-ffb001b78b47", 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 2, 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4300, 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T07:05:00.224 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 31.373941061, 2024-04-22T07:05:00.225 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-04-22T07:05:00.225 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 25, 2024-04-22T07:05:00.225 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 23, 2024-04-22T07:05:00.225 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 53.08997643 2024-04-22T07:05:00.225 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:05:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 -- 172.21.15.44:0/3832304637 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fba8406c8f0 msgr2=0x7fba8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fba8406c8f0 0x7fba8406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba8c001eb0 tx=0x7fba8c009040 comp rx=0 tx=0).stop 2024-04-22T07:05:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 -- 172.21.15.44:0/3832304637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa41464c0 msgr2=0x7fbaa4142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa41464c0 0x7fbaa4142680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00e200 tx=0x7fba9c009750 comp rx=0 tx=0).stop 2024-04-22T07:05:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 -- 172.21.15.44:0/3832304637 shutdown_connections 2024-04-22T07:05:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fba8406c8f0 0x7fba8406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fbaa41464c0 0x7fbaa4142680 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaa40c2e10 0x7fbaa41ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.226 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 --2- 172.21.15.44:0/3832304637 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fba900051c0 0x7fbaa414f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 -- 172.21.15.44:0/3832304637 >> 172.21.15.44:0/3832304637 conn(0x7fbaa406b970 msgr2=0x7fbaa406bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 -- 172.21.15.44:0/3832304637 shutdown_connections 2024-04-22T07:05:00.227 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.221+0000 7fbaa954b640 1 -- 172.21.15.44:0/3832304637 wait complete. 2024-04-22T07:05:00.229 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T07:05:00.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.333+0000 7f76bd45b640 1 Processor -- start 2024-04-22T07:05:00.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.333+0000 7f76bd45b640 1 -- start start 2024-04-22T07:05:00.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76bd45b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b80c57d0 0x7f76b80c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76bd45b640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b80cbd70 con 0x7f76b8155010 2024-04-22T07:05:00.333 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76bd45b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b815b780 con 0x7f76b80cc980 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76bd45b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f76b815ba20 con 0x7f76b80c57d0 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b6ffd640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f76b8155010 0x7f76b806b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60768/0 (socket says 172.21.15.44:60768) 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b6ffd640 1 -- 172.21.15.44:0/2316058058 learned_addr learned my addr 172.21.15.44:0/2316058058 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b67fc640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b80c57d0 0x7f76b80c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 125795105 0 0) 0x7f76a8001110 con 0x7f76b8155010 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a0003520 con 0x7f76b8155010 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3429956259 0 0) 0x7f76b815b780 con 0x7f76b80cc980 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a8001110 con 0x7f76b80cc980 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 <== mon.0 v1:172.21.15.44:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 328575008 0 0) 0x7f76a0003520 con 0x7f76b8155010 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b80c57d0 msgr2=0x7f76b80c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.334 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b80c57d0 0x7f76b80c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 >> v1:172.21.15.70:6789/0 conn(0x7f76b80cc980 legacy=0x7f76b81655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 --> v1:172.21.15.44:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b80cb590 con 0x7f76b8155010 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 <== mon.0 v1:172.21.15.44:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3718881788 0 0) 0x7f76a8001840 con 0x7f76b8155010 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b67fc640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b80c57d0 0x7f76b80c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 >> v1:172.21.15.44:6789/0 conn(0x7f76b8155010 legacy=0x7f76b806b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b80c57d0 0x7f76a0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 0x7f76a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a0004d40 0x7f76a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76b815ba20 con 0x7f76a0004100 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f76a000bad0 con 0x7f76a0004d40 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.334+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f76a000bcd0 con 0x7f76b80c57d0 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.335+0000 7f76b67fc640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 0x7f76a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.335 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.335+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b80c57d0 0x7f76a0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.335+0000 7f76b77fe640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a0004d40 0x7f76a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.335+0000 7f76b67fc640 1 -- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b80c57d0 msgr2=0x7f76a0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.335+0000 7f76b67fc640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b80c57d0 0x7f76a0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.335+0000 7f76b67fc640 1 -- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a0004d40 msgr2=0x7f76a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.335+0000 7f76b67fc640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a0004d40 0x7f76a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.335+0000 7f76b67fc640 1 -- 172.21.15.44:0/2316058058 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a0003520 con 0x7f76a0004100 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.335+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b80c57d0 0x7f76a0003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76b67fc640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 0x7f76a0004800 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009c40 tx=0x7f76a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975d9f36298b5010 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac039070 con 0x7f76a0004100 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76ac02c8d0 con 0x7f76a0004100 2024-04-22T07:05:00.336 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76b5ffb640 1 -- 172.21.15.44:0/2316058058 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac009840 con 0x7f76a0004100 2024-04-22T07:05:00.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76bd45b640 1 -- 172.21.15.44:0/2316058058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 msgr2=0x7f76a0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76bd45b640 1 --2- 172.21.15.44:0/2316058058 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 0x7f76a0004800 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f76ac009c40 tx=0x7f76a0005830 comp rx=0 tx=0).stop 2024-04-22T07:05:00.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76b77fe640 1 -- 172.21.15.44:0/2316058058 reap_dead start 2024-04-22T07:05:00.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76bd45b640 1 -- 172.21.15.44:0/2316058058 shutdown_connections 2024-04-22T07:05:00.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76bd45b640 1 -- 172.21.15.44:0/2316058058 >> 172.21.15.44:0/2316058058 conn(0x7f76b8067df0 msgr2=0x7f76b80d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76bd45b640 1 -- 172.21.15.44:0/2316058058 shutdown_connections 2024-04-22T07:05:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.336+0000 7f76bd45b640 1 -- 172.21.15.44:0/2316058058 wait complete. 2024-04-22T07:05:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76bd45b640 1 Processor -- start 2024-04-22T07:05:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76bd45b640 1 -- start start 2024-04-22T07:05:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76bd45b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 0x7f76b8158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76bd45b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b8150730 0x7f76b814a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76bd45b640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b814a860 0x7f76b8159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76bd45b640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76b815e550 con 0x7f76a0004100 2024-04-22T07:05:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76bd45b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f76b815e270 con 0x7f76b8150730 2024-04-22T07:05:00.340 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76bd45b640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f76b815e830 con 0x7f76b814a860 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76b6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 0x7f76b8158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76b6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 0x7f76b8158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37098/0 (socket says 172.21.15.44:37098) 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76b6ffd640 1 -- 172.21.15.44:0/3969098898 learned_addr learned my addr 172.21.15.44:0/3969098898 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76b67fc640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b8150730 0x7f76b814a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.338+0000 7f76b77fe640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b814a860 0x7f76b8159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f76b6ffd640 1 -- 172.21.15.44:0/3969098898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b814a860 msgr2=0x7f76b8159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b814a860 0x7f76b8159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f76b6ffd640 1 -- 172.21.15.44:0/3969098898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b8150730 msgr2=0x7f76b814a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b8150730 0x7f76b814a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f76b6ffd640 1 -- 172.21.15.44:0/3969098898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b8150f20 con 0x7f76a0004100 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 0x7f76b8158240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f76a800e950 tx=0x7f76a800ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f769f7fe640 1 -- 172.21.15.44:0/3969098898 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8019070 con 0x7f76a0004100 2024-04-22T07:05:00.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f769f7fe640 1 -- 172.21.15.44:0/3969098898 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a800c8d0 con 0x7f76a0004100 2024-04-22T07:05:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f769f7fe640 1 -- 172.21.15.44:0/3969098898 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a800cc80 con 0x7f76a0004100 2024-04-22T07:05:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f76bd45b640 1 -- 172.21.15.44:0/3969098898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b81f0e70 con 0x7f76a0004100 2024-04-22T07:05:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.339+0000 7f76bd45b640 1 -- 172.21.15.44:0/3969098898 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81f1410 con 0x7f76a0004100 2024-04-22T07:05:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.341+0000 7f769f7fe640 1 -- 172.21.15.44:0/3969098898 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76a801c070 con 0x7f76a0004100 2024-04-22T07:05:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.341+0000 7f769f7fe640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f76a406c8f0 0x7f76a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.341+0000 7f769f7fe640 1 -- 172.21.15.44:0/3969098898 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f76a808b430 con 0x7f76a0004100 2024-04-22T07:05:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.341+0000 7f76b67fc640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f76a406c8f0 0x7f76a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.342+0000 7f76b67fc640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f76a406c8f0 0x7f76a406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f76ac036000 tx=0x7f76ac030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.342+0000 7f76c0045740 1 Processor -- start 2024-04-22T07:05:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.342+0000 7f76c0045740 1 -- start start 2024-04-22T07:05:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76c0045740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 0x563d61a0ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76c0045740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c6ec0 0x563d61a0f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76c0045740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c2ca0 0x563d619eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.348 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76c0045740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563d61929b10 con 0x563d619c2ca0 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76c0045740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563d618c7240 con 0x563d619c6ec0 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76c0045740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563d61925ad0 con 0x563d61a05e40 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c2ca0 0x563d619eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b77fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c2ca0 0x563d619eaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37110/0 (socket says 172.21.15.44:37110) 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b77fe640 1 -- 172.21.15.44:0/4154605933 learned_addr learned my addr 172.21.15.44:0/4154605933 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/4154605933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 0x563d61a0ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b67fc640 1 --2- 172.21.15.44:0/4154605933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c6ec0 0x563d61a0f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b77fe640 1 -- 172.21.15.44:0/4154605933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 msgr2=0x563d61a0ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b77fe640 1 --2- 172.21.15.44:0/4154605933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 0x563d61a0ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b77fe640 1 -- 172.21.15.44:0/4154605933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c6ec0 msgr2=0x563d61a0f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b77fe640 1 --2- 172.21.15.44:0/4154605933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c6ec0 0x563d61a0f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b77fe640 1 -- 172.21.15.44:0/4154605933 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6163c2d0 con 0x563d619c2ca0 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.343+0000 7f76b67fc640 1 --2- 172.21.15.44:0/4154605933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c6ec0 0x563d61a0f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:00.349 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.344+0000 7f76b77fe640 1 --2- 172.21.15.44:0/4154605933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c2ca0 0x563d619eaee0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f76b80c7ce0 tx=0x7f76b0012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3150c9ca989ff8d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.344+0000 7f769cff9640 1 -- 172.21.15.44:0/4154605933 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0012ea0 con 0x563d619c2ca0 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.344+0000 7f769cff9640 1 -- 172.21.15.44:0/4154605933 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b0006b50 con 0x563d619c2ca0 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.344+0000 7f769cff9640 1 -- 172.21.15.44:0/4154605933 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0006d00 con 0x563d619c2ca0 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.344+0000 7f76c0045740 1 -- 172.21.15.44:0/4154605933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c2ca0 msgr2=0x563d619eaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.344+0000 7f76c0045740 1 --2- 172.21.15.44:0/4154605933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c2ca0 0x563d619eaee0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f76b80c7ce0 tx=0x7f76b0012690 comp rx=0 tx=0).stop 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.344+0000 7f76c0045740 1 -- 172.21.15.44:0/4154605933 shutdown_connections 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.345+0000 7f76c0045740 1 --2- 172.21.15.44:0/4154605933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c2ca0 0x563d619eaee0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.345+0000 7f76c0045740 1 --2- 172.21.15.44:0/4154605933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c6ec0 0x563d61a0f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.345+0000 7f76c0045740 1 --2- 172.21.15.44:0/4154605933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 0x563d61a0ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.345+0000 7f76c0045740 1 -- 172.21.15.44:0/4154605933 >> 172.21.15.44:0/4154605933 conn(0x563d619c1990 msgr2=0x563d619c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.345+0000 7f76c0045740 1 -- 172.21.15.44:0/4154605933 shutdown_connections 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.345+0000 7f76c0045740 1 -- 172.21.15.44:0/4154605933 wait complete. 2024-04-22T07:05:00.350 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.345+0000 7f76c0045740 1 -- start start 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.345+0000 7f76c0045740 1 Processor -- start 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76c0045740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d619c6ec0 0x563d619dc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76c0045740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d61a05e40 0x563d619be170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76c0045740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619be6b0 0x563d61a08240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76c0045740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563d619c5250 con 0x563d61a05e40 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76c0045740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563d618b2190 con 0x563d619be6b0 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76c0045740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563d619c5530 con 0x563d619c6ec0 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76c0045740 10 client.0 _ll_register_callbacks cb 0x563d618ad8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76b67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d61a05e40 0x563d619be170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76b67fc640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d61a05e40 0x563d619be170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37122/0 (socket says 172.21.15.44:37122) 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76b67fc640 1 -- 172.21.15.44:0/3238784316 learned_addr learned my addr 172.21.15.44:0/3238784316 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76b77fe640 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619be6b0 0x563d61a08240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d619c6ec0 0x563d619dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.351 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76b67fc640 1 -- 172.21.15.44:0/3238784316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d619c6ec0 msgr2=0x563d619dc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76b67fc640 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d619c6ec0 0x563d619dc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76b67fc640 1 -- 172.21.15.44:0/3238784316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619be6b0 msgr2=0x563d61a08240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76b67fc640 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619be6b0 0x563d61a08240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.346+0000 7f76b67fc640 1 -- 172.21.15.44:0/3238784316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d617066b0 con 0x563d61a05e40 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76b67fc640 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d61a05e40 0x563d619be170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f76ac02c480 tx=0x7f76ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392989ed47972516 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f7695ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f7695ffb640 1 -- 172.21.15.44:0/3238784316 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac050060 con 0x563d61a05e40 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f7695ffb640 1 -- 172.21.15.44:0/3238784316 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f76ac034910 con 0x563d61a05e40 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76c0045740 1 -- 172.21.15.44:0/3238784316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b0006b50 con 0x563d61a05e40 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f7695ffb640 1 -- 172.21.15.44:0/3238784316 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f76ac039040 con 0x563d61a05e40 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76c0045740 10 client.4944 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76c0045740 1 -- 172.21.15.44:0/3238784316 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x563d61a077e0 con 0x563d61a05e40 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f7695ffb640 1 -- 172.21.15.44:0/3238784316 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76b0006b50 con 0x563d61a05e40 2024-04-22T07:05:00.352 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f7695ffb640 1 -- 172.21.15.44:0/3238784316 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f76ac059080 con 0x563d61a05e40 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76c0045740 10 client.4944 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76c0045740 10 client.4944 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76c0045740 20 client.4944 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76c0045740 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x563d619ff8d0 0x563d61a19da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76c0045740 4 client.4944 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76c0045740 1 -- 172.21.15.44:0/3238784316 --> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d61937530 con 0x563d619ff8d0 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.347+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x563d619ff8d0 0x563d61a19da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f7695ffb640 10 client.4944 ms_handle_connect on v2:172.21.15.44:6837/1911557586 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x563d619ff8d0 0x563d61a19da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94fceda9febd1582 server_cookie=e9d2fc790e32a7f5 in_seq=0 out_seq=0 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f7695ffb640 1 -- 172.21.15.44:0/3238784316 <== mds.1 v2:172.21.15.44:6837/1911557586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d61937530 con 0x563d619ff8d0 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f7695ffb640 10 client.4944 handle_command_reply: tid=0 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f76c0045740 1 client.4944 shutdown 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f76c0045740 1 -- 172.21.15.44:0/3238784316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d61a05e40 msgr2=0x563d619be170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f76c0045740 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d61a05e40 0x563d619be170 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f76ac02c480 tx=0x7f76ac0046c0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.353 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f76c0045740 1 -- 172.21.15.44:0/3238784316 shutdown_connections 2024-04-22T07:05:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f76c0045740 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x563d619ff8d0 0x563d61a19da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f76c0045740 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619be6b0 0x563d61a08240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f76c0045740 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d61a05e40 0x563d619be170 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f76c0045740 1 --2- 172.21.15.44:0/3238784316 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d619c6ec0 0x563d619dc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f7695ffb640 0 client.4944 ms_handle_reset on v2:172.21.15.44:6837/1911557586 2024-04-22T07:05:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.348+0000 7f76c0045740 1 -- 172.21.15.44:0/3238784316 >> 172.21.15.44:0/3238784316 conn(0x563d619c2ca0 msgr2=0x563d61a0eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.349+0000 7f76c0045740 1 -- 172.21.15.44:0/3238784316 shutdown_connections 2024-04-22T07:05:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.349+0000 7f76c0045740 1 -- 172.21.15.44:0/3238784316 wait complete. 2024-04-22T07:05:00.354 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.349+0000 7f76c0045740 20 client.4944 trim_cache size 0 max 16384 2024-04-22T07:05:00.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.367+0000 7f76c0045740 1 Processor -- start 2024-04-22T07:05:00.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.367+0000 7f76c0045740 1 -- start start 2024-04-22T07:05:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76c0045740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c6ec0 0x563d61a065e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76c0045740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 0x563d61a06b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76c0045740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c48e0 0x563d619e54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76c0045740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f76b815d8e0 con 0x563d619c6ec0 2024-04-22T07:05:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76c0045740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563d619e6430 con 0x563d619c48e0 2024-04-22T07:05:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76c0045740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563d619ff750 con 0x563d61a05e40 2024-04-22T07:05:00.367 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c6ec0 0x563d61a065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c6ec0 0x563d61a065e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37128/0 (socket says 172.21.15.44:37128) 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b6ffd640 1 -- 172.21.15.44:0/215155668 learned_addr learned my addr 172.21.15.44:0/215155668 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b67fc640 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 0x563d61a06b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b77fe640 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c48e0 0x563d619e54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b6ffd640 1 -- 172.21.15.44:0/215155668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 msgr2=0x563d61a06b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 0x563d61a06b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b6ffd640 1 -- 172.21.15.44:0/215155668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c48e0 msgr2=0x563d619e54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c48e0 0x563d619e54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b6ffd640 1 -- 172.21.15.44:0/215155668 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d619e65b0 con 0x563d619c6ec0 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b67fc640 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 0x563d61a06b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b77fe640 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c48e0 0x563d619e54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c6ec0 0x563d61a065e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f76a8059120 tx=0x7f76a805c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4afbd07a88a4132 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f7697fff640 1 -- 172.21.15.44:0/215155668 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8057230 con 0x563d619c6ec0 2024-04-22T07:05:00.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f7697fff640 1 -- 172.21.15.44:0/215155668 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a8057490 con 0x563d619c6ec0 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.368+0000 7f7697fff640 1 -- 172.21.15.44:0/215155668 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8055800 con 0x563d619c6ec0 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.369+0000 7f76c0045740 1 -- 172.21.15.44:0/215155668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c6ec0 msgr2=0x563d61a065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.369+0000 7f76c0045740 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c6ec0 0x563d61a065e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f76a8059120 tx=0x7f76a805c040 comp rx=0 tx=0).stop 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.369+0000 7f76c0045740 1 -- 172.21.15.44:0/215155668 shutdown_connections 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.369+0000 7f76c0045740 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d61a05e40 0x563d61a06b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.369+0000 7f76c0045740 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d619c48e0 0x563d619e54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.369+0000 7f76c0045740 1 --2- 172.21.15.44:0/215155668 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c6ec0 0x563d61a065e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.369+0000 7f76c0045740 1 -- 172.21.15.44:0/215155668 >> 172.21.15.44:0/215155668 conn(0x563d619c2ca0 msgr2=0x563d61a09030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.369+0000 7f76c0045740 1 -- 172.21.15.44:0/215155668 shutdown_connections 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.369+0000 7f76c0045740 1 -- 172.21.15.44:0/215155668 wait complete. 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.369+0000 7f76c0045740 1 -- start start 2024-04-22T07:05:00.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76c0045740 1 Processor -- start 2024-04-22T07:05:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76c0045740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c48e0 0x563d619c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76c0045740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d619c6ec0 0x563d61a7ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76c0045740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d61a05e40 0x563d61a7b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76c0045740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x563d61a19c20 con 0x563d619c48e0 2024-04-22T07:05:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76c0045740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x563d619ffce0 con 0x563d61a05e40 2024-04-22T07:05:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76c0045740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x563d61a1a2f0 con 0x563d619c6ec0 2024-04-22T07:05:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76c0045740 10 client.0 _ll_register_callbacks cb 0x7f76a00036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:00.373 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76b6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c48e0 0x563d619c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76b6ffd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c48e0 0x563d619c3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37144/0 (socket says 172.21.15.44:37144) 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76b6ffd640 1 -- 172.21.15.44:0/2331356095 learned_addr learned my addr 172.21.15.44:0/2331356095 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76b67fc640 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d619c6ec0 0x563d61a7ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.370+0000 7f76b77fe640 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d61a05e40 0x563d61a7b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76b6ffd640 1 -- 172.21.15.44:0/2331356095 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d619c6ec0 msgr2=0x563d61a7ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d619c6ec0 0x563d61a7ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76b6ffd640 1 -- 172.21.15.44:0/2331356095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d61a05e40 msgr2=0x563d61a7b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d61a05e40 0x563d61a7b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76b6ffd640 1 -- 172.21.15.44:0/2331356095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d619c07d0 con 0x563d619c48e0 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76b6ffd640 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c48e0 0x563d619c3860 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f76a800c4b0 tx=0x7f76a808b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f5dc6992df720a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f7695ffb640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f7695ffb640 1 -- 172.21.15.44:0/2331356095 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8055800 con 0x563d619c48e0 2024-04-22T07:05:00.374 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f7695ffb640 1 -- 172.21.15.44:0/2331356095 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f76a808b0b0 con 0x563d619c48e0 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76c0045740 1 -- 172.21.15.44:0/2331356095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76a8057490 con 0x563d619c48e0 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f7695ffb640 1 -- 172.21.15.44:0/2331356095 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f76a80559a0 con 0x563d619c48e0 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76c0045740 10 client.4950 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76c0045740 1 -- 172.21.15.44:0/2331356095 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x563d619ba9a0 con 0x563d619c48e0 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f7695ffb640 1 -- 172.21.15.44:0/2331356095 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a8057490 con 0x563d619c48e0 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f7695ffb640 1 -- 172.21.15.44:0/2331356095 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f76a805d070 con 0x563d619c48e0 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76c0045740 10 client.4950 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76c0045740 10 client.4950 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76c0045740 20 client.4950 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76c0045740 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x563d61a02980 0x563d61a02d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76c0045740 4 client.4950 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.371+0000 7f76c0045740 1 -- 172.21.15.44:0/2331356095 --> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] -- command(tid 0: {"prefix": "status"}) -- 0x563d619313a0 con 0x563d61a02980 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76b67fc640 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x563d61a02980 0x563d61a02d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f7695ffb640 10 client.4950 ms_handle_connect on v2:172.21.15.44:6837/1911557586 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76b67fc640 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x563d61a02980 0x563d61a02d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75e141cc57caef2 server_cookie=149849fee1e3377d in_seq=0 out_seq=0 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f7695ffb640 1 -- 172.21.15.44:0/2331356095 <== mds.1 v2:172.21.15.44:6837/1911557586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563d619313a0 con 0x563d61a02980 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f7695ffb640 10 client.4950 handle_command_reply: tid=0 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 client.4950 shutdown 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 -- 172.21.15.44:0/2331356095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c48e0 msgr2=0x563d619c3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c48e0 0x563d619c3860 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f76a800c4b0 tx=0x7f76a808b040 comp rx=0 tx=0).stop 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 -- 172.21.15.44:0/2331356095 shutdown_connections 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.44:6837/1911557586,v1:172.21.15.44:6839/1911557586] conn(0x563d61a02980 0x563d61a02d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x563d619c6ec0 0x563d61a7ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.376 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d61a05e40 0x563d61a7b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f7695ffb640 0 client.4950 ms_handle_reset on v2:172.21.15.44:6837/1911557586 2024-04-22T07:05:00.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 --2- 172.21.15.44:0/2331356095 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x563d619c48e0 0x563d619c3860 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 -- 172.21.15.44:0/2331356095 >> 172.21.15.44:0/2331356095 conn(0x563d619c2ca0 msgr2=0x563d61a08f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 -- 172.21.15.44:0/2331356095 shutdown_connections 2024-04-22T07:05:00.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 1 -- 172.21.15.44:0/2331356095 wait complete. 2024-04-22T07:05:00.377 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.372+0000 7f76c0045740 20 client.4950 trim_cache size 0 max 16384 2024-04-22T07:05:00.377 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:05:00.377 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "dc21e26c-6c65-477f-a367-ffb001b78b47", 2024-04-22T07:05:00.377 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 1, 2024-04-22T07:05:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4303, 2024-04-22T07:05:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T07:05:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T07:05:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T07:05:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 37.549411374000002, 2024-04-22T07:05:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-04-22T07:05:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 25, 2024-04-22T07:05:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 23, 2024-04-22T07:05:00.378 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 53.241311723000003 2024-04-22T07:05:00.378 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:05:00.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 -- 172.21.15.44:0/3969098898 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f76a406c8f0 msgr2=0x7f76a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f76a406c8f0 0x7f76a406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f76ac036000 tx=0x7f76ac030000 comp rx=0 tx=0).stop 2024-04-22T07:05:00.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 -- 172.21.15.44:0/3969098898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 msgr2=0x7f76b8158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 0x7f76b8158240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f76a800e950 tx=0x7f76a800ee10 comp rx=0 tx=0).stop 2024-04-22T07:05:00.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 -- 172.21.15.44:0/3969098898 shutdown_connections 2024-04-22T07:05:00.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f76a406c8f0 0x7f76a406edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f76b814a860 0x7f76b8159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76b8150730 0x7f76b814a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.379 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 --2- 172.21.15.44:0/3969098898 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f76a0004100 0x7f76b8158240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 -- 172.21.15.44:0/3969098898 >> 172.21.15.44:0/3969098898 conn(0x7f76b8067df0 msgr2=0x7f76b806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 -- 172.21.15.44:0/3969098898 shutdown_connections 2024-04-22T07:05:00.380 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.373+0000 7f76bd45b640 1 -- 172.21.15.44:0/3969098898 wait complete. 2024-04-22T07:05:00.381 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-22T07:05:00.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.485+0000 7ff511fca640 1 Processor -- start 2024-04-22T07:05:00.483 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.485+0000 7ff511fca640 1 -- start start 2024-04-22T07:05:00.484 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.486+0000 7ff511fca640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 0x7ff50c0c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.486+0000 7ff511fca640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff50c06b970 0x7ff50c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.486+0000 7ff511fca640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff50c15e550 con 0x7ff50c0ce850 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.486+0000 7ff511fca640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7ff50c15e810 con 0x7ff50c06b970 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.486+0000 7ff511fca640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c15eaf0 con 0x7ff50c155120 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.486+0000 7ff50affd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff50c06b970 0x7ff50c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.486+0000 7ff50bfff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff50c0ce850 0x7ff50c15b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:60780/0 (socket says 172.21.15.44:60780) 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.486+0000 7ff50bfff640 1 -- 172.21.15.44:0/563097933 learned_addr learned my addr 172.21.15.44:0/563097933 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.486+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/563097933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 0x7ff50c0c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.487+0000 7ff50affd640 1 -- 172.21.15.44:0/563097933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 msgr2=0x7ff50c0c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.487+0000 7ff50affd640 1 --2- 172.21.15.44:0/563097933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 0x7ff50c0c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.487+0000 7ff50affd640 1 -- 172.21.15.44:0/563097933 >> v1:172.21.15.44:6789/0 conn(0x7ff50c0ce850 legacy=0x7ff50c15b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.487+0000 7ff50affd640 1 -- 172.21.15.44:0/563097933 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c15edd0 con 0x7ff50c06b970 2024-04-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.487+0000 7ff50a7fc640 1 -- 172.21.15.44:0/563097933 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1991838277 0 0) 0x7ff50c15e550 con 0x7ff50c0ce850 2024-04-22T07:05:00.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.487+0000 7ff50affd640 1 --2- 172.21.15.44:0/563097933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 0x7ff50c0ce310 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5000097a0 tx=0x7ff50002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf4e9e3c87c5ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff50a7fc640 1 -- 172.21.15.44:0/563097933 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500035020 con 0x7ff50c06b970 2024-04-22T07:05:00.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff50a7fc640 1 -- 172.21.15.44:0/563097933 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff500031830 con 0x7ff50c06b970 2024-04-22T07:05:00.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff50a7fc640 1 -- 172.21.15.44:0/563097933 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500031c00 con 0x7ff50c06b970 2024-04-22T07:05:00.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff511fca640 1 -- 172.21.15.44:0/563097933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 msgr2=0x7ff50c0ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff511fca640 1 --2- 172.21.15.44:0/563097933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 0x7ff50c0ce310 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5000097a0 tx=0x7ff50002b8d0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff511fca640 1 -- 172.21.15.44:0/563097933 shutdown_connections 2024-04-22T07:05:00.487 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff511fca640 1 --2- 172.21.15.44:0/563097933 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 0x7ff50c0ce310 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff511fca640 1 --2- 172.21.15.44:0/563097933 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 0x7ff50c0c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff511fca640 1 -- 172.21.15.44:0/563097933 >> 172.21.15.44:0/563097933 conn(0x7ff50c0713b0 msgr2=0x7ff50c0717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff511fca640 1 -- 172.21.15.44:0/563097933 shutdown_connections 2024-04-22T07:05:00.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.488+0000 7ff511fca640 1 -- 172.21.15.44:0/563097933 wait complete. 2024-04-22T07:05:00.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.489+0000 7ff511fca640 1 Processor -- start 2024-04-22T07:05:00.488 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.489+0000 7ff511fca640 1 -- start start 2024-04-22T07:05:00.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.490+0000 7ff511fca640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 0x7ff50c150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.490+0000 7ff511fca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff50c0ce850 0x7ff50c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.490+0000 7ff511fca640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 0x7ff50c1f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.490+0000 7ff511fca640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c06cdc0 con 0x7ff50c06b970 2024-04-22T07:05:00.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.490+0000 7ff511fca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c0c92a0 con 0x7ff50c0ce850 2024-04-22T07:05:00.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.490+0000 7ff511fca640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c06cf40 con 0x7ff50c155120 2024-04-22T07:05:00.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.490+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 0x7ff50c150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.490+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 0x7ff50c150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37156/0 (socket says 172.21.15.44:37156) 2024-04-22T07:05:00.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.490+0000 7ff50b7fe640 1 -- 172.21.15.44:0/4177171416 learned_addr learned my addr 172.21.15.44:0/4177171416 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.490 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff50b7fe640 1 -- 172.21.15.44:0/4177171416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 msgr2=0x7ff50c1f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff50bfff640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 0x7ff50c1f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff50affd640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff50c0ce850 0x7ff50c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 0x7ff50c1f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff50b7fe640 1 -- 172.21.15.44:0/4177171416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff50c0ce850 msgr2=0x7ff50c153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff50c0ce850 0x7ff50c153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff50b7fe640 1 -- 172.21.15.44:0/4177171416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c158770 con 0x7ff50c06b970 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff50bfff640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 0x7ff50c1f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff50affd640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff50c0ce850 0x7ff50c153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 0x7ff50c150ed0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00e870 tx=0x7ff4fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff4f3fff640 1 -- 172.21.15.44:0/4177171416 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc017070 con 0x7ff50c06b970 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff4f3fff640 1 -- 172.21.15.44:0/4177171416 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4fc00c940 con 0x7ff50c06b970 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff511fca640 1 -- 172.21.15.44:0/4177171416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c155840 con 0x7ff50c06b970 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff4f3fff640 1 -- 172.21.15.44:0/4177171416 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00cd30 con 0x7ff50c06b970 2024-04-22T07:05:00.491 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.491+0000 7ff511fca640 1 -- 172.21.15.44:0/4177171416 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c150960 con 0x7ff50c06b970 2024-04-22T07:05:00.493 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.493+0000 7ff4f3fff640 1 -- 172.21.15.44:0/4177171416 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4fc01a070 con 0x7ff50c06b970 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.493+0000 7ff4f3fff640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4f806c920 0x7ff4f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.493+0000 7ff50affd640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4f806c920 0x7ff4f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.493+0000 7ff4f3fff640 1 -- 172.21.15.44:0/4177171416 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7ff4fc08ba10 con 0x7ff50c06b970 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.494+0000 7ff50affd640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4f806c920 0x7ff4f806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff500036000 tx=0x7ff500008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.494+0000 7ff514bb6740 1 Processor -- start 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.494+0000 7ff514bb6740 1 -- start start 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad23f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 0x555d9ad244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd7ca0 0x555d9acffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff514bb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555d9ac3eb10 con 0x555d9acd7ca0 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff514bb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555d9abdc240 con 0x555d9ad1ae40 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff514bb6740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555d9ac3aad0 con 0x555d9acdbec0 2024-04-22T07:05:00.494 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd7ca0 0x555d9acffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff50bfff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd7ca0 0x555d9acffee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37170/0 (socket says 172.21.15.44:37170) 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff50bfff640 1 -- 172.21.15.44:0/1853961088 learned_addr learned my addr 172.21.15.44:0/1853961088 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff50affd640 1 --2- 172.21.15.44:0/1853961088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 0x555d9ad244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/1853961088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad23f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff50bfff640 1 -- 172.21.15.44:0/1853961088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 msgr2=0x555d9ad244a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff50bfff640 1 --2- 172.21.15.44:0/1853961088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 0x555d9ad244a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff50bfff640 1 -- 172.21.15.44:0/1853961088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 msgr2=0x555d9ad23f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff50bfff640 1 --2- 172.21.15.44:0/1853961088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad23f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.495+0000 7ff50bfff640 1 -- 172.21.15.44:0/1853961088 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9a9512d0 con 0x555d9acd7ca0 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.496+0000 7ff50bfff640 1 --2- 172.21.15.44:0/1853961088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd7ca0 0x555d9acffee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff504001e90 tx=0x7ff50400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d9100feff841a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.496+0000 7ff4f17fa640 1 -- 172.21.15.44:0/1853961088 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504004d60 con 0x555d9acd7ca0 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.496+0000 7ff4f17fa640 1 -- 172.21.15.44:0/1853961088 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5040020a0 con 0x555d9acd7ca0 2024-04-22T07:05:00.495 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.496+0000 7ff4f17fa640 1 -- 172.21.15.44:0/1853961088 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040072d0 con 0x555d9acd7ca0 2024-04-22T07:05:00.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.496+0000 7ff514bb6740 1 -- 172.21.15.44:0/1853961088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd7ca0 msgr2=0x555d9acffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.500 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.496+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1853961088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd7ca0 0x555d9acffee0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff504001e90 tx=0x7ff50400d520 comp rx=0 tx=0).stop 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.497+0000 7ff514bb6740 1 -- 172.21.15.44:0/1853961088 shutdown_connections 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.497+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1853961088 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 0x555d9ad244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.497+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1853961088 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd7ca0 0x555d9acffee0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.497+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1853961088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad23f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.497+0000 7ff514bb6740 1 -- 172.21.15.44:0/1853961088 >> 172.21.15.44:0/1853961088 conn(0x555d9acd6990 msgr2=0x555d9acd9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.497+0000 7ff514bb6740 1 -- 172.21.15.44:0/1853961088 shutdown_connections 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.497+0000 7ff514bb6740 1 -- 172.21.15.44:0/1853961088 wait complete. 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.497+0000 7ff514bb6740 1 -- start start 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.497+0000 7ff514bb6740 1 Processor -- start 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9acd3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9ad1ae40 0x555d9ad2f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.501 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9acd52a0 0x555d9ad2f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff514bb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555d9acda250 con 0x555d9acdbec0 2024-04-22T07:05:00.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff514bb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555d9abc7190 con 0x555d9acd52a0 2024-04-22T07:05:00.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff514bb6740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555d9acda530 con 0x555d9ad1ae40 2024-04-22T07:05:00.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff514bb6740 10 client.0 _ll_register_callbacks cb 0x555d9abc28b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:00.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9acd3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9acd3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37182/0 (socket says 172.21.15.44:37182) 2024-04-22T07:05:00.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50b7fe640 1 -- 172.21.15.44:0/2031642141 learned_addr learned my addr 172.21.15.44:0/2031642141 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.502 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50affd640 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9ad1ae40 0x555d9ad2f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50bfff640 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9acd52a0 0x555d9ad2f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50b7fe640 1 -- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9ad1ae40 msgr2=0x555d9ad2f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9ad1ae40 0x555d9ad2f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50b7fe640 1 -- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9acd52a0 msgr2=0x555d9ad2f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9acd52a0 0x555d9ad2f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50b7fe640 1 -- 172.21.15.44:0/2031642141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d9aa1b6b0 con 0x555d9acdbec0 2024-04-22T07:05:00.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9acd3170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc055bb0 tx=0x7ff4fc059f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafe03da6969afd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.498+0000 7ff4ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:00.503 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/2031642141 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc088060 con 0x555d9acdbec0 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/2031642141 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7ff4fc058ba0 con 0x555d9acdbec0 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff514bb6740 1 -- 172.21.15.44:0/2031642141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d9ac4c530 con 0x555d9acdbec0 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/2031642141 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff4fc021040 con 0x555d9acdbec0 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff514bb6740 10 client.4965 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff514bb6740 1 -- 172.21.15.44:0/2031642141 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x555d9ad20360 con 0x555d9acdbec0 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/2031642141 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555d9ac4c530 con 0x555d9acdbec0 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/2031642141 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7ff4fc024060 con 0x555d9acdbec0 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff514bb6740 10 client.4965 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff514bb6740 10 client.4965 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff514bb6740 20 client.4965 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff514bb6740 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x555d9acf9f70 0x555d9ad3aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.504 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff514bb6740 4 client.4965 mds_command: new command op to 4324 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff514bb6740 1 -- 172.21.15.44:0/2031642141 --> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d9ac463a0 con 0x555d9acf9f70 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.499+0000 7ff50affd640 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x555d9acf9f70 0x555d9ad3aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.500+0000 7ff4ea7fc640 10 client.4965 ms_handle_connect on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.500+0000 7ff50affd640 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x555d9acf9f70 0x555d9ad3aad0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c86cec49ac6a0cd server_cookie=9a963c259036fd73 in_seq=0 out_seq=0 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.500+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/2031642141 <== mds.0 v2:172.21.15.70:6836/2872131879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d9ac463a0 con 0x555d9acf9f70 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff4ea7fc640 10 client.4965 handle_command_reply: tid=0 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 client.4965 shutdown 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 -- 172.21.15.44:0/2031642141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 msgr2=0x555d9acd3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9acd3170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc055bb0 tx=0x7ff4fc059f10 comp rx=0 tx=0).stop 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 -- 172.21.15.44:0/2031642141 shutdown_connections 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9ad1ae40 0x555d9ad2f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x555d9acf9f70 0x555d9ad3aad0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.505 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9acd52a0 0x555d9ad2f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff4ea7fc640 0 client.4965 ms_handle_reset on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:00.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 --2- 172.21.15.44:0/2031642141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9acd3170 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 -- 172.21.15.44:0/2031642141 >> 172.21.15.44:0/2031642141 conn(0x555d9acd7ca0 msgr2=0x555d9acdaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 -- 172.21.15.44:0/2031642141 shutdown_connections 2024-04-22T07:05:00.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 1 -- 172.21.15.44:0/2031642141 wait complete. 2024-04-22T07:05:00.506 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.501+0000 7ff514bb6740 20 client.4965 trim_cache size 0 max 16384 2024-04-22T07:05:00.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.519+0000 7ff514bb6740 1 Processor -- start 2024-04-22T07:05:00.517 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.519+0000 7ff514bb6740 1 -- start start 2024-04-22T07:05:00.519 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd52a0 0x555d9ad24030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 0x555d9ad18090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad185d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff514bb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1fda10 con 0x555d9acd52a0 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff514bb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1fcc70 con 0x555d9ad1ae40 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff514bb6740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c1fd340 con 0x555d9acdbec0 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd52a0 0x555d9ad24030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd52a0 0x555d9ad24030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37184/0 (socket says 172.21.15.44:37184) 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50b7fe640 1 -- 172.21.15.44:0/1674407616 learned_addr learned my addr 172.21.15.44:0/1674407616 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50bfff640 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad185d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50affd640 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 0x555d9ad18090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50b7fe640 1 -- 172.21.15.44:0/1674407616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 msgr2=0x555d9ad18090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 0x555d9ad18090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.520 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50b7fe640 1 -- 172.21.15.44:0/1674407616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 msgr2=0x555d9ad185d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad185d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50b7fe640 1 -- 172.21.15.44:0/1674407616 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9acf1b90 con 0x555d9acd52a0 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50bfff640 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad185d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.520+0000 7ff50affd640 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 0x555d9ad18090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd52a0 0x555d9ad24030 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc061e00 tx=0x7ff4fc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b073ed63d3ff8d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff4f0ff9640 1 -- 172.21.15.44:0/1674407616 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0577f0 con 0x555d9acd52a0 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff4f0ff9640 1 -- 172.21.15.44:0/1674407616 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4fc058ba0 con 0x555d9acd52a0 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff4f0ff9640 1 -- 172.21.15.44:0/1674407616 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0554a0 con 0x555d9acd52a0 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff514bb6740 1 -- 172.21.15.44:0/1674407616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd52a0 msgr2=0x555d9ad24030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd52a0 0x555d9ad24030 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc061e00 tx=0x7ff4fc01e000 comp rx=0 tx=0).stop 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff514bb6740 1 -- 172.21.15.44:0/1674407616 shutdown_connections 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acdbec0 0x555d9ad18090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad185d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.521 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1674407616 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acd52a0 0x555d9ad24030 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff514bb6740 1 -- 172.21.15.44:0/1674407616 >> 172.21.15.44:0/1674407616 conn(0x555d9acd7ca0 msgr2=0x555d9ad20f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff514bb6740 1 -- 172.21.15.44:0/1674407616 shutdown_connections 2024-04-22T07:05:00.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.521+0000 7ff514bb6740 1 -- 172.21.15.44:0/1674407616 wait complete. 2024-04-22T07:05:00.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.522+0000 7ff514bb6740 1 -- start start 2024-04-22T07:05:00.522 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.522+0000 7ff514bb6740 1 Processor -- start 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acd52a0 0x555d9acd8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9ad3f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff514bb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad8fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff514bb6740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x555d9ad20970 con 0x555d9acdbec0 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff514bb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c1f0dc0 con 0x555d9ad1ae40 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff514bb6740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x555d9acfa480 con 0x555d9acd52a0 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff514bb6740 10 client.0 _ll_register_callbacks cb 0x7ff4fc0600b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9ad3f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9ad3f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37186/0 (socket says 172.21.15.44:37186) 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50affd640 1 -- 172.21.15.44:0/1898934359 learned_addr learned my addr 172.21.15.44:0/1898934359 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acd52a0 0x555d9acd8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.526 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50bfff640 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad8fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50affd640 1 -- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acd52a0 msgr2=0x555d9acd8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50affd640 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acd52a0 0x555d9acd8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50affd640 1 -- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 msgr2=0x555d9ad8fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50affd640 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad8fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50affd640 1 -- 172.21.15.44:0/1898934359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d9ad40460 con 0x555d9acdbec0 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acd52a0 0x555d9acd8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff50affd640 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9ad3f2d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff50002bee0 tx=0x7ff50002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc174d60eb95ee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.523+0000 7ff4ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/1898934359 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500059060 con 0x555d9acdbec0 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/1898934359 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7ff500054830 con 0x555d9acdbec0 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff514bb6740 1 -- 172.21.15.44:0/1898934359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc058ba0 con 0x555d9acdbec0 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/1898934359 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff50005f050 con 0x555d9acdbec0 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff514bb6740 10 client.4971 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:00.527 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff514bb6740 1 -- 172.21.15.44:0/1898934359 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x555d9acd3820 con 0x555d9acdbec0 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/1898934359 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4fc058ba0 con 0x555d9acdbec0 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/1898934359 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7ff500062020 con 0x555d9acdbec0 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff514bb6740 10 client.4971 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff514bb6740 10 client.4971 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff514bb6740 20 client.4971 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x555d9ad99b30 0x555d9ada2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff514bb6740 4 client.4971 mds_command: new command op to 4324 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff514bb6740 1 -- 172.21.15.44:0/1898934359 --> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] -- command(tid 0: {"prefix": "status"}) -- 0x555d9abc2960 con 0x555d9ad99b30 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.524+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x555d9ad99b30 0x555d9ada2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff50b7fe640 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x555d9ad99b30 0x555d9ada2600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ced82fee6ecfd7 server_cookie=1f80b6ea78cf4140 in_seq=0 out_seq=0 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff4ea7fc640 10 client.4971 ms_handle_connect on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff4ea7fc640 1 -- 172.21.15.44:0/1898934359 <== mds.0 v2:172.21.15.70:6836/2872131879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x555d9abc2960 con 0x555d9ad99b30 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff4ea7fc640 10 client.4971 handle_command_reply: tid=0 2024-04-22T07:05:00.528 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 client.4971 shutdown 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 -- 172.21.15.44:0/1898934359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 msgr2=0x555d9ad3f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9ad3f2d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff50002bee0 tx=0x7ff50002e000 comp rx=0 tx=0).stop 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 -- 172.21.15.44:0/1898934359 shutdown_connections 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x555d9ad99b30 0x555d9ada2600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555d9ad1ae40 0x555d9ad8fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555d9acdbec0 0x555d9ad3f2d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 --2- 172.21.15.44:0/1898934359 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x555d9acd52a0 0x555d9acd8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 -- 172.21.15.44:0/1898934359 >> 172.21.15.44:0/1898934359 conn(0x555d9acd7ca0 msgr2=0x555d9acf84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff4ea7fc640 0 client.4971 ms_handle_reset on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 -- 172.21.15.44:0/1898934359 shutdown_connections 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 1 -- 172.21.15.44:0/1898934359 wait complete. 2024-04-22T07:05:00.529 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.525+0000 7ff514bb6740 20 client.4971 trim_cache size 0 max 16384 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "cluster_fsid": "dc21e26c-6c65-477f-a367-ffb001b78b47", 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "whoami": 0, 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "id": 4324, 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "want_state": "up:active", 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "state": "up:active", 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "fs_name": "cephfs", 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "rank_uptime": 46.733596007999999, 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "mdsmap_epoch": 17, 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch": 25, 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "osdmap_epoch_barrier": 23, 2024-04-22T07:05:00.530 INFO:teuthology.orchestra.run.smithi044.stdout: "uptime": 53.379883694999997 2024-04-22T07:05:00.531 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:05:00.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 -- 172.21.15.44:0/4177171416 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4f806c920 msgr2=0x7ff4f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4f806c920 0x7ff4f806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff500036000 tx=0x7ff500008030 comp rx=0 tx=0).stop 2024-04-22T07:05:00.531 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 -- 172.21.15.44:0/4177171416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 msgr2=0x7ff50c150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:00.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 0x7ff50c150ed0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00e870 tx=0x7ff4fc00ed30 comp rx=0 tx=0).stop 2024-04-22T07:05:00.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 -- 172.21.15.44:0/4177171416 shutdown_connections 2024-04-22T07:05:00.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4f806c920 0x7ff4f806ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff50c155120 0x7ff50c1f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff50c0ce850 0x7ff50c153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 --2- 172.21.15.44:0/4177171416 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff50c06b970 0x7ff50c150ed0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:00.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 -- 172.21.15.44:0/4177171416 >> 172.21.15.44:0/4177171416 conn(0x7ff50c0713b0 msgr2=0x7ff50c158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:00.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 -- 172.21.15.44:0/4177171416 shutdown_connections 2024-04-22T07:05:00.532 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:00.526+0000 7ff511fca640 1 -- 172.21.15.44:0/4177171416 wait complete. 2024-04-22T07:05:00.534 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-22T07:05:10.609 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:05:10.612 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:05:15.535 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-04-22T07:05:15.535 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:05:15.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.650+0000 7fdf4c832640 1 Processor -- start 2024-04-22T07:05:15.649 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.650+0000 7fdf4c832640 1 -- start start 2024-04-22T07:05:15.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.651+0000 7fdf4c832640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf480c87f0 0x7fdf480c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:15.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.651+0000 7fdf4c832640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf48158960 0x7fdf48158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:15.650 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.651+0000 7fdf4c832640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf4815cd80 con 0x7fdf48152d50 2024-04-22T07:05:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.651+0000 7fdf4c832640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf48153130 con 0x7fdf480c87f0 2024-04-22T07:05:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.651+0000 7fdf4c832640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdf480cc380 con 0x7fdf48158960 2024-04-22T07:05:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.651+0000 7fdf45d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf480c87f0 0x7fdf480c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.651+0000 7fdf45d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf480c87f0 0x7fdf480c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37604/0 (socket says 172.21.15.44:37604) 2024-04-22T07:05:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.651+0000 7fdf45d9b640 1 -- 172.21.15.44:0/3277418440 learned_addr learned my addr 172.21.15.44:0/3277418440 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.651+0000 7fdf46d9d640 1 --2- 172.21.15.44:0/3277418440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf48158960 0x7fdf48158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.652+0000 7fdf45d9b640 1 -- 172.21.15.44:0/3277418440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf48158960 msgr2=0x7fdf48158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.652+0000 7fdf45d9b640 1 --2- 172.21.15.44:0/3277418440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf48158960 0x7fdf48158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.652+0000 7fdf45d9b640 1 -- 172.21.15.44:0/3277418440 >> v1:172.21.15.70:6789/0 conn(0x7fdf48152d50 legacy=0x7fdf480c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:15.651 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.652+0000 7fdf45d9b640 1 -- 172.21.15.44:0/3277418440 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf480c6c10 con 0x7fdf480c87f0 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf45d9b640 1 --2- 172.21.15.44:0/3277418440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf480c87f0 0x7fdf480c9750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009870 tx=0x7fdf3002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfa86be9362235f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4559a640 1 -- 172.21.15.44:0/3277418440 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3002f840 con 0x7fdf480c87f0 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4559a640 1 -- 172.21.15.44:0/3277418440 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3002f9e0 con 0x7fdf480c87f0 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4559a640 1 -- 172.21.15.44:0/3277418440 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3002fdb0 con 0x7fdf480c87f0 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4c832640 1 -- 172.21.15.44:0/3277418440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf480c87f0 msgr2=0x7fdf480c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4c832640 1 --2- 172.21.15.44:0/3277418440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf480c87f0 0x7fdf480c9750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009870 tx=0x7fdf3002ba20 comp rx=0 tx=0).stop 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4c832640 1 -- 172.21.15.44:0/3277418440 shutdown_connections 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4c832640 1 --2- 172.21.15.44:0/3277418440 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf48158960 0x7fdf48158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4c832640 1 --2- 172.21.15.44:0/3277418440 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf480c87f0 0x7fdf480c9750 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4c832640 1 -- 172.21.15.44:0/3277418440 >> 172.21.15.44:0/3277418440 conn(0x7fdf4806d870 msgr2=0x7fdf4806dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:15.653 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4c832640 1 -- 172.21.15.44:0/3277418440 shutdown_connections 2024-04-22T07:05:15.654 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.653+0000 7fdf4c832640 1 -- 172.21.15.44:0/3277418440 wait complete. 2024-04-22T07:05:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.655+0000 7fdf4c832640 1 Processor -- start 2024-04-22T07:05:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.655+0000 7fdf4c832640 1 -- start start 2024-04-22T07:05:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.655+0000 7fdf4c832640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480c87f0 0x7fdf48157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.655+0000 7fdf4c832640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf48152d50 0x7fdf48154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:15.656 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf4c832640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf48158960 0x7fdf48154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf4c832640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fdf480cccf0 con 0x7fdf48152d50 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf4c832640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdf480cca10 con 0x7fdf480c87f0 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf4c832640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdf481592d0 con 0x7fdf48158960 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf45d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf48152d50 0x7fdf48154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf45d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf48152d50 0x7fdf48154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37618/0 (socket says 172.21.15.44:37618) 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf45d9b640 1 -- 172.21.15.44:0/3748794921 learned_addr learned my addr 172.21.15.44:0/3748794921 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf4659c640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480c87f0 0x7fdf48157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf46d9d640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf48158960 0x7fdf48154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf45d9b640 1 -- 172.21.15.44:0/3748794921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf48158960 msgr2=0x7fdf48154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf45d9b640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf48158960 0x7fdf48154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:15.657 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf45d9b640 1 -- 172.21.15.44:0/3748794921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480c87f0 msgr2=0x7fdf48157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:15.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf45d9b640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480c87f0 0x7fdf48157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:15.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf45d9b640 1 -- 172.21.15.44:0/3748794921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf481eefa0 con 0x7fdf48152d50 2024-04-22T07:05:15.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf45d9b640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf48152d50 0x7fdf48154320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf3002ba60 tx=0x7fdf30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:15.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf3effd640 1 -- 172.21.15.44:0/3748794921 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30047070 con 0x7fdf48152d50 2024-04-22T07:05:15.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf3effd640 1 -- 172.21.15.44:0/3748794921 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf30037bc0 con 0x7fdf48152d50 2024-04-22T07:05:15.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf3effd640 1 -- 172.21.15.44:0/3748794921 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30042480 con 0x7fdf48152d50 2024-04-22T07:05:15.658 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.656+0000 7fdf4c832640 1 -- 172.21.15.44:0/3748794921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf481ef240 con 0x7fdf48152d50 2024-04-22T07:05:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.657+0000 7fdf4c832640 1 -- 172.21.15.44:0/3748794921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf48153840 con 0x7fdf48152d50 2024-04-22T07:05:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.658+0000 7fdf4c832640 1 -- 172.21.15.44:0/3748794921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4804fdc0 con 0x7fdf48152d50 2024-04-22T07:05:15.659 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.658+0000 7fdf3effd640 1 -- 172.21.15.44:0/3748794921 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf3004a070 con 0x7fdf48152d50 2024-04-22T07:05:15.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.659+0000 7fdf3effd640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdf2806c8f0 0x7fdf2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:15.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.659+0000 7fdf3effd640 1 -- 172.21.15.44:0/3748794921 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7fdf300adc10 con 0x7fdf48152d50 2024-04-22T07:05:15.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.659+0000 7fdf4659c640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdf2806c8f0 0x7fdf2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:15.660 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.659+0000 7fdf4659c640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdf2806c8f0 0x7fdf2806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf40001780 tx=0x7fdf40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:15.662 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.663+0000 7fdf3effd640 1 -- 172.21.15.44:0/3748794921 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf30033030 con 0x7fdf48152d50 2024-04-22T07:05:15.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.890+0000 7fdf4c832640 1 -- 172.21.15.44:0/3748794921 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf480cc750 con 0x7fdf48152d50 2024-04-22T07:05:15.889 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.891+0000 7fdf3effd640 1 -- 172.21.15.44:0/3748794921 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6908 (secure 0 0 0) 0x7fdf300b1020 con 0x7fdf48152d50 2024-04-22T07:05:15.890 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:05:15.890 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:04:44.948538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4324,"mds_1":4303,"mds_2":4300,"mds_3":4278,"mds_4":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"b","rank":3,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4324,"qdb_cluster":[4275,4278,4300,4303,4324]},"id":1}]} 2024-04-22T07:05:15.890 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-04-22T07:05:15.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 -- 172.21.15.44:0/3748794921 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdf2806c8f0 msgr2=0x7fdf2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdf2806c8f0 0x7fdf2806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdf40001780 tx=0x7fdf40009040 comp rx=0 tx=0).stop 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 -- 172.21.15.44:0/3748794921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf48152d50 msgr2=0x7fdf48154320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf48152d50 0x7fdf48154320 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf3002ba60 tx=0x7fdf30004730 comp rx=0 tx=0).stop 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 -- 172.21.15.44:0/3748794921 shutdown_connections 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fdf2806c8f0 0x7fdf2806edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdf48158960 0x7fdf48154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fdf48152d50 0x7fdf48154320 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 --2- 172.21.15.44:0/3748794921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480c87f0 0x7fdf48157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 -- 172.21.15.44:0/3748794921 >> 172.21.15.44:0/3748794921 conn(0x7fdf4806d870 msgr2=0x7fdf480ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 -- 172.21.15.44:0/3748794921 shutdown_connections 2024-04-22T07:05:15.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:15.893+0000 7fdf4c832640 1 -- 172.21.15.44:0/3748794921 wait complete. 2024-04-22T07:05:15.903 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.4324 quiesce db / --timeout 60 --expiration 79.4 --await' 2024-04-22T07:05:15.903 DEBUG:teuthology.orchestra.run.smithi044:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4324 quiesce db / --timeout 60 --expiration 79.4 --await 2024-04-22T07:05:16.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.013+0000 7f2338e93640 1 Processor -- start 2024-04-22T07:05:16.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.013+0000 7f2338e93640 1 -- start start 2024-04-22T07:05:16.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f2338e93640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2334154f80 0x7f233406c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f2338e93640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f23340cbca0 0x7f23340d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f2338e93640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f233415eca0 con 0x7f23340cc940 2024-04-22T07:05:16.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f2338e93640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2334155360 con 0x7f2334154f80 2024-04-22T07:05:16.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f2338e93640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f233415ef40 con 0x7f23340cbca0 2024-04-22T07:05:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f233259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2334154f80 0x7f233406c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f233259c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2334154f80 0x7f233406c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37628/0 (socket says 172.21.15.44:37628) 2024-04-22T07:05:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f233259c640 1 -- 172.21.15.44:0/2000816501 learned_addr learned my addr 172.21.15.44:0/2000816501 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f2331d9b640 1 --2- 172.21.15.44:0/2000816501 >> v2:172.21.15.70:3300/0 conn(0x7f23340cbca0 0x7f23340d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f233259c640 1 -- 172.21.15.44:0/2000816501 >> v2:172.21.15.70:3300/0 conn(0x7f23340cbca0 msgr2=0x7f23340d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f233259c640 1 --2- 172.21.15.44:0/2000816501 >> v2:172.21.15.70:3300/0 conn(0x7f23340cbca0 0x7f23340d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f233259c640 1 -- 172.21.15.44:0/2000816501 >> v1:172.21.15.70:6789/0 conn(0x7f23340cc940 legacy=0x7f233415aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:16.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.014+0000 7f233259c640 1 -- 172.21.15.44:0/2000816501 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23340cafc0 con 0x7f2334154f80 2024-04-22T07:05:16.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.015+0000 7f233259c640 1 --2- 172.21.15.44:0/2000816501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2334154f80 0x7f233406c530 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f23340cdf80 tx=0x7f232c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb084f313122bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:16.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.015+0000 7f233159a640 1 -- 172.21.15.44:0/2000816501 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c02f850 con 0x7f2334154f80 2024-04-22T07:05:16.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.015+0000 7f233159a640 1 -- 172.21.15.44:0/2000816501 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f232c02f9f0 con 0x7f2334154f80 2024-04-22T07:05:16.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.015+0000 7f233159a640 1 -- 172.21.15.44:0/2000816501 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c02fdc0 con 0x7f2334154f80 2024-04-22T07:05:16.015 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.016+0000 7f2338e93640 1 -- 172.21.15.44:0/2000816501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2334154f80 msgr2=0x7f233406c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.016+0000 7f2338e93640 1 --2- 172.21.15.44:0/2000816501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2334154f80 0x7f233406c530 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f23340cdf80 tx=0x7f232c02ba30 comp rx=0 tx=0).stop 2024-04-22T07:05:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.016+0000 7f2338e93640 1 -- 172.21.15.44:0/2000816501 shutdown_connections 2024-04-22T07:05:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.016+0000 7f2338e93640 1 --2- 172.21.15.44:0/2000816501 >> v2:172.21.15.70:3300/0 conn(0x7f23340cbca0 0x7f23340d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.016+0000 7f2338e93640 1 --2- 172.21.15.44:0/2000816501 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f2334154f80 0x7f233406c530 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.016+0000 7f2338e93640 1 -- 172.21.15.44:0/2000816501 >> 172.21.15.44:0/2000816501 conn(0x7f2334073470 msgr2=0x7f2334073880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.016+0000 7f2338e93640 1 -- 172.21.15.44:0/2000816501 shutdown_connections 2024-04-22T07:05:16.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.016+0000 7f2338e93640 1 -- 172.21.15.44:0/2000816501 wait complete. 2024-04-22T07:05:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.017+0000 7f2338e93640 1 Processor -- start 2024-04-22T07:05:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.017+0000 7f2338e93640 1 -- start start 2024-04-22T07:05:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f2338e93640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23340cbca0 0x7f23340b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f2338e93640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23340cc940 0x7f23340c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f2338e93640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2334154f80 0x7f23340c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f2338e93640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f233415b5c0 con 0x7f23340cbca0 2024-04-22T07:05:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f2338e93640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f23340cc170 con 0x7f23340cc940 2024-04-22T07:05:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f2338e93640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f233415b740 con 0x7f2334154f80 2024-04-22T07:05:16.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f233259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23340cbca0 0x7f23340b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f233259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23340cbca0 0x7f23340b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37634/0 (socket says 172.21.15.44:37634) 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f233259c640 1 -- 172.21.15.44:0/816115475 learned_addr learned my addr 172.21.15.44:0/816115475 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f2331d9b640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23340cc940 0x7f23340c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.018+0000 7f2332d9d640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2334154f80 0x7f23340c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f233259c640 1 -- 172.21.15.44:0/816115475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2334154f80 msgr2=0x7f23340c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f233259c640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2334154f80 0x7f23340c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f233259c640 1 -- 172.21.15.44:0/816115475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23340cc940 msgr2=0x7f23340c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f233259c640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23340cc940 0x7f23340c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f233259c640 1 -- 172.21.15.44:0/816115475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2334152bd0 con 0x7f23340cbca0 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f233259c640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23340cbca0 0x7f23340b52a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f232c02ba70 tx=0x7f232c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f232affd640 1 -- 172.21.15.44:0/816115475 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c037d10 con 0x7f23340cbca0 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f232affd640 1 -- 172.21.15.44:0/816115475 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f232c046070 con 0x7f23340cbca0 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f232affd640 1 -- 172.21.15.44:0/816115475 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c041620 con 0x7f23340cbca0 2024-04-22T07:05:16.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f2338e93640 1 -- 172.21.15.44:0/816115475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23341524e0 con 0x7f23340cbca0 2024-04-22T07:05:16.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.019+0000 7f2338e93640 1 -- 172.21.15.44:0/816115475 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23341f4570 con 0x7f23340cbca0 2024-04-22T07:05:16.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.021+0000 7f232affd640 1 -- 172.21.15.44:0/816115475 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f232c040070 con 0x7f23340cbca0 2024-04-22T07:05:16.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.021+0000 7f232affd640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f231406c920 0x7f231406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.021+0000 7f232affd640 1 -- 172.21.15.44:0/816115475 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f232c0ae1c0 con 0x7f23340cbca0 2024-04-22T07:05:16.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.021+0000 7f2331d9b640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f231406c920 0x7f231406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.022+0000 7f2331d9b640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f231406c920 0x7f231406ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f23340ca300 tx=0x7f231c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:16.021 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.022+0000 7f233bb67740 1 Processor -- start 2024-04-22T07:05:16.022 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.022+0000 7f233bb67740 1 -- start start 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f233bb67740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2caec200 0x556d2caec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f233bb67740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2caecb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f233bb67740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caeac60 0x556d2cb2bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f233bb67740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556d2ca4d0b0 con 0x556d2cb36080 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f233bb67740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x556d2c9fe680 con 0x556d2caec200 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f233bb67740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x556d2ca0ac60 con 0x556d2caeac60 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f2331d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2caecb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f2331d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2caecb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37642/0 (socket says 172.21.15.44:37642) 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f2331d9b640 1 -- 172.21.15.44:0/261619327 learned_addr learned my addr 172.21.15.44:0/261619327 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f233259c640 1 --2- 172.21.15.44:0/261619327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2caec200 0x556d2caec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f2332d9d640 1 --2- 172.21.15.44:0/261619327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caeac60 0x556d2cb2bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f2331d9b640 1 -- 172.21.15.44:0/261619327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caeac60 msgr2=0x556d2cb2bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:16.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f2331d9b640 1 --2- 172.21.15.44:0/261619327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caeac60 0x556d2cb2bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f2331d9b640 1 -- 172.21.15.44:0/261619327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2caec200 msgr2=0x556d2caec5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:16.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f2331d9b640 1 --2- 172.21.15.44:0/261619327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2caec200 0x556d2caec5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.023+0000 7f2331d9b640 1 -- 172.21.15.44:0/261619327 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2c82fc90 con 0x556d2cb36080 2024-04-22T07:05:16.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.024+0000 7f2331d9b640 1 --2- 172.21.15.44:0/261619327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2caecb20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f231c0109a0 tx=0x7f231c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1132a6f1e4153d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:16.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.024+0000 7f2313fff640 1 -- 172.21.15.44:0/261619327 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c01c050 con 0x556d2cb36080 2024-04-22T07:05:16.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.024+0000 7f2313fff640 1 -- 172.21.15.44:0/261619327 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c026070 con 0x556d2cb36080 2024-04-22T07:05:16.024 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.024+0000 7f2313fff640 1 -- 172.21.15.44:0/261619327 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c016e10 con 0x556d2cb36080 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.025+0000 7f233bb67740 1 -- 172.21.15.44:0/261619327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 msgr2=0x556d2caecb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.025+0000 7f233bb67740 1 --2- 172.21.15.44:0/261619327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2caecb20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f231c0109a0 tx=0x7f231c013000 comp rx=0 tx=0).stop 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.025+0000 7f233bb67740 1 -- 172.21.15.44:0/261619327 shutdown_connections 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.025+0000 7f233bb67740 1 --2- 172.21.15.44:0/261619327 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caeac60 0x556d2cb2bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.025+0000 7f233bb67740 1 --2- 172.21.15.44:0/261619327 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2caecb20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.025+0000 7f233bb67740 1 --2- 172.21.15.44:0/261619327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2caec200 0x556d2caec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.025+0000 7f233bb67740 1 -- 172.21.15.44:0/261619327 >> 172.21.15.44:0/261619327 conn(0x556d2cae8c00 msgr2=0x556d2caeb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.025+0000 7f233bb67740 1 -- 172.21.15.44:0/261619327 shutdown_connections 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.025+0000 7f233bb67740 1 -- 172.21.15.44:0/261619327 wait complete. 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.026+0000 7f233bb67740 1 -- start start 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.026+0000 7f233bb67740 1 Processor -- start 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.026+0000 7f233bb67740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 0x556d2cb3c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.027 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.026+0000 7f233bb67740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2cb144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.026+0000 7f233bb67740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb0d6d0 0x556d2cb2ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.026+0000 7f233bb67740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x556d2ca50400 con 0x556d2cb36080 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.026+0000 7f233bb67740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x556d2ca0c840 con 0x556d2cb0d6d0 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.026+0000 7f233bb67740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x556d2ca55720 con 0x556d2caec200 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f233bb67740 10 client.0 _ll_register_callbacks cb 0x556d2ca5dea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f2331d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2cb144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f2331d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2cb144c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37650/0 (socket says 172.21.15.44:37650) 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f2331d9b640 1 -- 172.21.15.44:0/722952149 learned_addr learned my addr 172.21.15.44:0/722952149 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f233259c640 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 0x556d2cb3c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f2332d9d640 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb0d6d0 0x556d2cb2ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f2331d9b640 1 -- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 msgr2=0x556d2cb3c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f2331d9b640 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 0x556d2cb3c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f2331d9b640 1 -- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb0d6d0 msgr2=0x556d2cb2ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:16.028 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f2331d9b640 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb0d6d0 0x556d2cb2ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f2331d9b640 1 -- 172.21.15.44:0/722952149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2c890420 con 0x556d2cb36080 2024-04-22T07:05:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f2331d9b640 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2cb144c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f231c023040 tx=0x7f231c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245cb39372b10bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f23117fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f23117fa640 1 -- 172.21.15.44:0/722952149 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c006240 con 0x556d2cb36080 2024-04-22T07:05:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f233bb67740 1 -- 172.21.15.44:0/722952149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f231c026070 con 0x556d2cb36080 2024-04-22T07:05:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f23117fa640 1 -- 172.21.15.44:0/722952149 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f231c0063e0 con 0x556d2cb36080 2024-04-22T07:05:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.027+0000 7f23117fa640 1 -- 172.21.15.44:0/722952149 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f231c03d050 con 0x556d2cb36080 2024-04-22T07:05:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.028+0000 7f233bb67740 10 client.4989 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:16.029 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.028+0000 7f233bb67740 1 -- 172.21.15.44:0/722952149 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x556d2c723580 con 0x556d2cb36080 2024-04-22T07:05:16.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.029+0000 7f23117fa640 1 -- 172.21.15.44:0/722952149 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c026070 con 0x556d2cb36080 2024-04-22T07:05:16.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.029+0000 7f23117fa640 1 -- 172.21.15.44:0/722952149 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f231c040020 con 0x556d2cb36080 2024-04-22T07:05:16.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.030+0000 7f233bb67740 10 client.4989 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:16.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.030+0000 7f233bb67740 10 client.4989 resolve_mds: validated gid 4324 aka daemon mds.f 2024-04-22T07:05:16.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.030+0000 7f233bb67740 20 client.4989 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:16.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.030+0000 7f233bb67740 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x556d2cb14f10 0x556d2cb270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.030+0000 7f233bb67740 4 client.4989 mds_command: new command op to 4324 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:05:16.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.030+0000 7f233bb67740 1 -- 172.21.15.44:0/722952149 --> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d2ca5eb10 con 0x556d2cb14f10 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.030+0000 7f233259c640 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x556d2cb14f10 0x556d2cb270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.030+0000 7f233259c640 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x556d2cb14f10 0x556d2cb270a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b95ad7119052eae0 server_cookie=e8cbaeec05671525 in_seq=0 out_seq=0 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.030+0000 7f23117fa640 10 client.4989 ms_handle_connect on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f23117fa640 1 -- 172.21.15.44:0/722952149 <== mds.0 v2:172.21.15.70:6836/2872131879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d2ca5eb10 con 0x556d2cb14f10 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f23117fa640 10 client.4989 handle_command_reply: tid=0 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 client.4989 shutdown 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 -- 172.21.15.44:0/722952149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 msgr2=0x556d2cb144c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2cb144c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f231c023040 tx=0x7f231c004730 comp rx=0 tx=0).stop 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 -- 172.21.15.44:0/722952149 shutdown_connections 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x556d2cb14f10 0x556d2cb270a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb0d6d0 0x556d2cb2ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb36080 0x556d2cb144c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f23117fa640 0 client.4989 ms_handle_reset on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:16.032 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 --2- 172.21.15.44:0/722952149 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 0x556d2cb3c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 -- 172.21.15.44:0/722952149 >> 172.21.15.44:0/722952149 conn(0x556d2caeac60 msgr2=0x556d2caea270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:16.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 -- 172.21.15.44:0/722952149 shutdown_connections 2024-04-22T07:05:16.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 1 -- 172.21.15.44:0/722952149 wait complete. 2024-04-22T07:05:16.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.031+0000 7f233bb67740 20 client.4989 trim_cache size 0 max 16384 2024-04-22T07:05:16.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f233bb67740 1 Processor -- start 2024-04-22T07:05:16.047 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f233bb67740 1 -- start start 2024-04-22T07:05:16.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f233bb67740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 0x556d2cae5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f233bb67740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cae5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f233bb67740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb36080 0x556d2cb273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f233bb67740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23341fe0e0 con 0x556d2cb0d6d0 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f233bb67740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f23341fc5a0 con 0x556d2cb36080 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f233bb67740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f23341fda10 con 0x556d2caec200 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f2331d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cae5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f2331d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cae5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37664/0 (socket says 172.21.15.44:37664) 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f2331d9b640 1 -- 172.21.15.44:0/1404169728 learned_addr learned my addr 172.21.15.44:0/1404169728 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.049+0000 7f233259c640 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 0x556d2cae5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f2332d9d640 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb36080 0x556d2cb273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f2331d9b640 1 -- 172.21.15.44:0/1404169728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 msgr2=0x556d2cae5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f2331d9b640 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 0x556d2cae5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f2331d9b640 1 -- 172.21.15.44:0/1404169728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb36080 msgr2=0x556d2cb273c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f2331d9b640 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb36080 0x556d2cb273c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f2331d9b640 1 -- 172.21.15.44:0/1404169728 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2c9f26f0 con 0x556d2cb0d6d0 2024-04-22T07:05:16.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f233259c640 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 0x556d2cae5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f2332d9d640 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb36080 0x556d2cb273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f2331d9b640 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cae5e80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f231c0073e0 tx=0x7f231c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c9fac6ddb8b6b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f23137fe640 1 -- 172.21.15.44:0/1404169728 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c01c030 con 0x556d2cb0d6d0 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f23137fe640 1 -- 172.21.15.44:0/1404169728 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c045020 con 0x556d2cb0d6d0 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.050+0000 7f23137fe640 1 -- 172.21.15.44:0/1404169728 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c005990 con 0x556d2cb0d6d0 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 -- 172.21.15.44:0/1404169728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 msgr2=0x556d2cae5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cae5e80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f231c0073e0 tx=0x7f231c02d070 comp rx=0 tx=0).stop 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 -- 172.21.15.44:0/1404169728 shutdown_connections 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2cb36080 0x556d2cb273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cae5e80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 --2- 172.21.15.44:0/1404169728 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2caec200 0x556d2cae5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 -- 172.21.15.44:0/1404169728 >> 172.21.15.44:0/1404169728 conn(0x556d2caeac60 msgr2=0x556d2cb3c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 -- 172.21.15.44:0/1404169728 shutdown_connections 2024-04-22T07:05:16.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 -- 172.21.15.44:0/1404169728 wait complete. 2024-04-22T07:05:16.054 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 -- start start 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.051+0000 7f233bb67740 1 Processor -- start 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f233bb67740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2caec200 0x556d2cb26a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f233bb67740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cb11d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f233bb67740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2cb36080 0x556d2cba0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f233bb67740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f23341fd340 con 0x556d2cb0d6d0 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f233bb67740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f23341fcc70 con 0x556d2caec200 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f233bb67740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f23341f0b80 con 0x556d2cb36080 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f233bb67740 10 client.0 _ll_register_callbacks cb 0x7f232c037eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f2331d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cb11d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f2331d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cb11d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:37674/0 (socket says 172.21.15.44:37674) 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f2331d9b640 1 -- 172.21.15.44:0/3119681537 learned_addr learned my addr 172.21.15.44:0/3119681537 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f233259c640 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2caec200 0x556d2cb26a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f2332d9d640 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2cb36080 0x556d2cba0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.055 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f2331d9b640 1 -- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2cb36080 msgr2=0x556d2cba0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f2331d9b640 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2cb36080 0x556d2cba0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f2331d9b640 1 -- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2caec200 msgr2=0x556d2cb26a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.052+0000 7f2331d9b640 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2caec200 0x556d2cb26a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f2331d9b640 1 -- 172.21.15.44:0/3119681537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2cb50050 con 0x556d2cb0d6d0 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f2331d9b640 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cb11d00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f231c016fd0 tx=0x7f231c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef807d91fb75543 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f23117fa640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f23117fa640 1 -- 172.21.15.44:0/3119681537 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0069a0 con 0x556d2cb0d6d0 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f23117fa640 1 -- 172.21.15.44:0/3119681537 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f231c035760 con 0x556d2cb0d6d0 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f233bb67740 1 -- 172.21.15.44:0/3119681537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f231c045020 con 0x556d2cb0d6d0 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f23117fa640 1 -- 172.21.15.44:0/3119681537 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f231c035980 con 0x556d2cb0d6d0 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f233bb67740 10 client.4995 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f233bb67740 1 -- 172.21.15.44:0/3119681537 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x556d2cb12e80 con 0x556d2cb0d6d0 2024-04-22T07:05:16.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f23117fa640 1 -- 172.21.15.44:0/3119681537 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f231c045020 con 0x556d2cb0d6d0 2024-04-22T07:05:16.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f23117fa640 1 -- 172.21.15.44:0/3119681537 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f231c040070 con 0x556d2cb0d6d0 2024-04-22T07:05:16.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f233bb67740 10 client.4995 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:16.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f233bb67740 10 client.4995 resolve_mds: validated gid 4324 aka daemon mds.f 2024-04-22T07:05:16.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f233bb67740 20 client.4995 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:16.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f233bb67740 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x556d2cb291b0 0x556d2cba0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:16.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f233bb67740 4 client.4995 mds_command: new command op to 4324 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.4, "await": true}] 2024-04-22T07:05:16.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f233bb67740 1 -- 172.21.15.44:0/3119681537 --> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.4, "await": true}) -- 0x556d2ca58980 con 0x556d2cb291b0 2024-04-22T07:05:16.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.053+0000 7f233259c640 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x556d2cb291b0 0x556d2cba0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:16.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.054+0000 7f233259c640 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x556d2cb291b0 0x556d2cba0c60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ef5eb456a8c749 server_cookie=3cbadd7aad5ab826 in_seq=0 out_seq=0 2024-04-22T07:05:16.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.054+0000 7f23117fa640 10 client.4995 ms_handle_connect on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:16.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.056+0000 7f23117fa640 1 -- 172.21.15.44:0/3119681537 <== mds.0 v2:172.21.15.70:6836/2872131879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+587 (crc 0 0 0) 0x556d2ca58980 con 0x556d2cb291b0 2024-04-22T07:05:16.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f23117fa640 10 client.4995 handle_command_reply: tid=0 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 client.4995 shutdown 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 -- 172.21.15.44:0/3119681537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 msgr2=0x556d2cb11d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cb11d00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f231c016fd0 tx=0x7f231c016000 comp rx=0 tx=0).stop 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 -- 172.21.15.44:0/3119681537 shutdown_connections 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x556d2cb36080 0x556d2cba0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x556d2cb291b0 0x556d2cba0c60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x556d2cb0d6d0 0x556d2cb11d00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 --2- 172.21.15.44:0/3119681537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556d2caec200 0x556d2cb26a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f23117fa640 0 client.4995 ms_handle_reset on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 -- 172.21.15.44:0/3119681537 >> 172.21.15.44:0/3119681537 conn(0x556d2caeac60 msgr2=0x556d2cb4f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 -- 172.21.15.44:0/3119681537 shutdown_connections 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 1 -- 172.21.15.44:0/3119681537 wait complete. 2024-04-22T07:05:16.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f233bb67740 20 client.4995 trim_cache size 0 max 16384 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 17, 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "leader": 4324, 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "set_version": 2, 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "sets": { 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "d430eb8b": { 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "version": 2, 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "age_ref": 0.0, 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "QUIESCED", 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T07:05:16.060 INFO:teuthology.orchestra.run.smithi044.stdout: "timeout": 60.0, 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: "expiration": 79.4, 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: "members": { 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: "file:/": { 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: "excluded": false, 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "QUIESCED", 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:05:16.061 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:05:16.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f2338e93640 1 -- 172.21.15.44:0/816115475 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f231406c920 msgr2=0x7f231406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:16.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f2338e93640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f231406c920 0x7f231406ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f23340ca300 tx=0x7f231c008040 comp rx=0 tx=0).stop 2024-04-22T07:05:16.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f2338e93640 1 -- 172.21.15.44:0/816115475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23340cbca0 msgr2=0x7f23340b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:16.062 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f2338e93640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23340cbca0 0x7f23340b52a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f232c02ba70 tx=0x7f232c004730 comp rx=0 tx=0).stop 2024-04-22T07:05:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f2338e93640 1 -- 172.21.15.44:0/816115475 shutdown_connections 2024-04-22T07:05:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f2338e93640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f231406c920 0x7f231406ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f2338e93640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2334154f80 0x7f23340c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f2338e93640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23340cc940 0x7f23340c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.057+0000 7f2338e93640 1 --2- 172.21.15.44:0/816115475 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f23340cbca0 0x7f23340b52a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.058+0000 7f2338e93640 1 -- 172.21.15.44:0/816115475 >> 172.21.15.44:0/816115475 conn(0x7f2334073470 msgr2=0x7f23340c5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.058+0000 7f2338e93640 1 -- 172.21.15.44:0/816115475 shutdown_connections 2024-04-22T07:05:16.063 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:16.058+0000 7f2338e93640 1 -- 172.21.15.44:0/816115475 wait complete. 2024-04-22T07:05:16.067 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: d430eb8b, quiesce took 0.5 seconds. Will release after: 19.4 2024-04-22T07:05:35.469 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:05:35.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.585+0000 7f8a9e87a640 1 Processor -- start 2024-04-22T07:05:35.583 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.585+0000 7f8a9e87a640 1 -- start start 2024-04-22T07:05:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.586+0000 7f8a9e87a640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8a98164630 0x7f8a98166a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.586+0000 7f8a9e87a640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a9815b4a0 con 0x7f8a980b3530 2024-04-22T07:05:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.586+0000 7f8a9e87a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a980701e0 con 0x7f8a980c8dd0 2024-04-22T07:05:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.586+0000 7f8a9e87a640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f8a980709a0 con 0x7f8a98164630 2024-04-22T07:05:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.586+0000 7f8a9ce17640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f8a98164630 0x7f8a98166a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.586+0000 7f8a97fff640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f8a980b3530 0x7f8a9815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:50212/0 (socket says 172.21.15.44:50212) 2024-04-22T07:05:35.584 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.586+0000 7f8a97fff640 1 -- 172.21.15.44:0/3926704606 learned_addr learned my addr 172.21.15.44:0/3926704606 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:05:35.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.587+0000 7f8a9ce17640 1 -- 172.21.15.44:0/3926704606 >> v1:172.21.15.70:6789/0 conn(0x7f8a980c8dd0 legacy=0x7f8a980c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.587+0000 7f8a9ce17640 1 -- 172.21.15.44:0/3926704606 >> v1:172.21.15.44:6789/0 conn(0x7f8a980b3530 legacy=0x7f8a9815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.587+0000 7f8a9ce17640 1 -- 172.21.15.44:0/3926704606 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9815e300 con 0x7f8a98164630 2024-04-22T07:05:35.585 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.587+0000 7f8a9ce17640 1 --2- 172.21.15.44:0/3926704606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a98164630 0x7f8a98166a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009d90 tx=0x7f8a9002bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572a91bbfae2b45a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.588+0000 7f8a96ffd640 1 -- 172.21.15.44:0/3926704606 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90039070 con 0x7f8a98164630 2024-04-22T07:05:35.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.588+0000 7f8a96ffd640 1 -- 172.21.15.44:0/3926704606 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a9002cd80 con 0x7f8a98164630 2024-04-22T07:05:35.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.588+0000 7f8a96ffd640 1 -- 172.21.15.44:0/3926704606 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90034660 con 0x7f8a98164630 2024-04-22T07:05:35.586 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.588+0000 7f8a9e87a640 1 -- 172.21.15.44:0/3926704606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a98164630 msgr2=0x7f8a98166a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.588+0000 7f8a9e87a640 1 --2- 172.21.15.44:0/3926704606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a98164630 0x7f8a98166a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009d90 tx=0x7f8a9002bfd0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.588+0000 7f8a9e87a640 1 -- 172.21.15.44:0/3926704606 shutdown_connections 2024-04-22T07:05:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.588+0000 7f8a9e87a640 1 --2- 172.21.15.44:0/3926704606 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a98164630 0x7f8a98166a20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.588+0000 7f8a9e87a640 1 -- 172.21.15.44:0/3926704606 >> 172.21.15.44:0/3926704606 conn(0x7f8a98071480 msgr2=0x7f8a98071890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.588+0000 7f8a9e87a640 1 -- 172.21.15.44:0/3926704606 shutdown_connections 2024-04-22T07:05:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.588+0000 7f8a9e87a640 1 -- 172.21.15.44:0/3926704606 wait complete. 2024-04-22T07:05:35.587 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.589+0000 7f8a9e87a640 1 Processor -- start 2024-04-22T07:05:35.588 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.590+0000 7f8a9e87a640 1 -- start start 2024-04-22T07:05:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.590+0000 7f8a9e87a640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a980b3530 0x7f8a980c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.590+0000 7f8a9e87a640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a980c8dd0 0x7f8a980c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.590+0000 7f8a9e87a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a98164630 0x7f8a98155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.590+0000 7f8a9e87a640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9815e8c0 con 0x7f8a980c8dd0 2024-04-22T07:05:35.589 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.590+0000 7f8a9e87a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9815ec30 con 0x7f8a98164630 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.590+0000 7f8a9e87a640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f8a980b3910 con 0x7f8a980b3530 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.590+0000 7f8a977fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a980c8dd0 0x7f8a980c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.590+0000 7f8a977fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a980c8dd0 0x7f8a980c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34386/0 (socket says 172.21.15.44:34386) 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a977fe640 1 -- 172.21.15.44:0/865170694 learned_addr learned my addr 172.21.15.44:0/865170694 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a97fff640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a980b3530 0x7f8a980c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a9ce17640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a98164630 0x7f8a98155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a977fe640 1 -- 172.21.15.44:0/865170694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a980b3530 msgr2=0x7f8a980c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a977fe640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a980b3530 0x7f8a980c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a977fe640 1 -- 172.21.15.44:0/865170694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a98164630 msgr2=0x7f8a98155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a977fe640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a98164630 0x7f8a98155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a977fe640 1 -- 172.21.15.44:0/865170694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a981f0d00 con 0x7f8a980c8dd0 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a977fe640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a980c8dd0 0x7f8a980c77d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000c440 tx=0x7f8a80001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.590 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a94ff9640 1 -- 172.21.15.44:0/865170694 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800052d0 con 0x7f8a980c8dd0 2024-04-22T07:05:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a94ff9640 1 -- 172.21.15.44:0/865170694 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a80005470 con 0x7f8a980c8dd0 2024-04-22T07:05:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a94ff9640 1 -- 172.21.15.44:0/865170694 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80007240 con 0x7f8a980c8dd0 2024-04-22T07:05:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a9e87a640 1 -- 172.21.15.44:0/865170694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a981f0fe0 con 0x7f8a980c8dd0 2024-04-22T07:05:35.591 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.591+0000 7f8a9e87a640 1 -- 172.21.15.44:0/865170694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a981f15a0 con 0x7f8a980c8dd0 2024-04-22T07:05:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.592+0000 7f8a9e87a640 1 -- 172.21.15.44:0/865170694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9815e0c0 con 0x7f8a980c8dd0 2024-04-22T07:05:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.593+0000 7f8a94ff9640 1 -- 172.21.15.44:0/865170694 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a800073e0 con 0x7f8a980c8dd0 2024-04-22T07:05:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.594+0000 7f8a94ff9640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8a7806c8f0 0x7f8a7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.594+0000 7f8a94ff9640 1 -- 172.21.15.44:0/865170694 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f8a8008afc0 con 0x7f8a980c8dd0 2024-04-22T07:05:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.594+0000 7f8a97fff640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8a7806c8f0 0x7f8a7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.592 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.594+0000 7f8a97fff640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8a7806c8f0 0x7f8a7806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8a88005880 tx=0x7f8a88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.598+0000 7f8a94ff9640 1 -- 172.21.15.44:0/865170694 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a80058d10 con 0x7f8a980c8dd0 2024-04-22T07:05:35.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.823+0000 7f8a9e87a640 1 -- 172.21.15.44:0/865170694 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a98070b60 con 0x7f8a980c8dd0 2024-04-22T07:05:35.821 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.824+0000 7f8a94ff9640 1 -- 172.21.15.44:0/865170694 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6908 (secure 0 0 0) 0x7f8a8005d010 con 0x7f8a980c8dd0 2024-04-22T07:05:35.822 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:05:35.822 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:04:44.948538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4324,"mds_1":4303,"mds_2":4300,"mds_3":4278,"mds_4":4275},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4275":{"gid":4275,"name":"d","rank":4,"incarnation":16,"state":"up:active","state_seq":11,"addr":"172.21.15.70:6834/1969018412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1969018412},{"type":"v1","addr":"172.21.15.70:6834","nonce":1969018412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"b","rank":3,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4324,"qdb_cluster":[4275,4278,4300,4303,4324]},"id":1}]} 2024-04-22T07:05:35.822 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 17 2024-04-22T07:05:35.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 -- 172.21.15.44:0/865170694 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8a7806c8f0 msgr2=0x7f8a7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8a7806c8f0 0x7f8a7806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8a88005880 tx=0x7f8a88009040 comp rx=0 tx=0).stop 2024-04-22T07:05:35.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 -- 172.21.15.44:0/865170694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a980c8dd0 msgr2=0x7f8a980c77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a980c8dd0 0x7f8a980c77d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000c440 tx=0x7f8a80001e70 comp rx=0 tx=0).stop 2024-04-22T07:05:35.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 -- 172.21.15.44:0/865170694 shutdown_connections 2024-04-22T07:05:35.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f8a7806c8f0 0x7f8a7806edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a98164630 0x7f8a98155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f8a980c8dd0 0x7f8a980c77d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.824 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 --2- 172.21.15.44:0/865170694 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f8a980b3530 0x7f8a980c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 -- 172.21.15.44:0/865170694 >> 172.21.15.44:0/865170694 conn(0x7f8a98071480 msgr2=0x7f8a980cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 -- 172.21.15.44:0/865170694 shutdown_connections 2024-04-22T07:05:35.825 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.826+0000 7f8a9e87a640 1 -- 172.21.15.44:0/865170694 wait complete. 2024-04-22T07:05:35.835 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.4324 quiesce db --set-id d430eb8b --release --await' 2024-04-22T07:05:35.835 DEBUG:teuthology.orchestra.run.smithi044:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4324 quiesce db --set-id d430eb8b --release --await 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.947+0000 7f5911a3e640 1 Processor -- start 2024-04-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.947+0000 7f5911a3e640 1 -- start start 2024-04-22T07:05:35.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f5911a3e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c1552c0 0x7f590c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.946 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f5911a3e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f590c0caec0 0x7f590c0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f5911a3e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f590c1556a0 con 0x7f590c0cc920 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f5911a3e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f590c15b160 con 0x7f590c0caec0 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f5911a3e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f590c15b440 con 0x7f590c1552c0 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f590b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f590c0cc920 0x7f590c0cd310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:51206/0 (socket says 172.21.15.44:51206) 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f590affd640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c1552c0 0x7f590c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f590a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f590c0caec0 0x7f590c0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f590b7fe640 1 -- 172.21.15.44:0/4290742522 learned_addr learned my addr 172.21.15.44:0/4290742522 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f5909ffb640 1 -- 172.21.15.44:0/4290742522 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4212817227 0 0) 0x7f590c1556a0 con 0x7f590c0cc920 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.948+0000 7f5909ffb640 1 -- 172.21.15.44:0/4290742522 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58f8003520 con 0x7f590c0cc920 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.949+0000 7f590affd640 1 -- 172.21.15.44:0/4290742522 >> v2:172.21.15.70:3300/0 conn(0x7f590c0caec0 msgr2=0x7f590c0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.949+0000 7f590affd640 1 --2- 172.21.15.44:0/4290742522 >> v2:172.21.15.70:3300/0 conn(0x7f590c0caec0 0x7f590c0d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.949+0000 7f590affd640 1 -- 172.21.15.44:0/4290742522 >> v1:172.21.15.70:6789/0 conn(0x7f590c0cc920 legacy=0x7f590c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.949+0000 7f590affd640 1 -- 172.21.15.44:0/4290742522 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590c15b720 con 0x7f590c1552c0 2024-04-22T07:05:35.947 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.949+0000 7f590a7fc640 1 --2- 172.21.15.44:0/4290742522 >> v2:172.21.15.70:3300/0 conn(0x7f590c0caec0 0x7f590c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:05:35.948 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.950+0000 7f590affd640 1 --2- 172.21.15.44:0/4290742522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c1552c0 0x7f590c0b32a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f58f40097a0 tx=0x7f58f402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bc5e7070c6f243a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.950+0000 7f5909ffb640 1 -- 172.21.15.44:0/4290742522 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4035020 con 0x7f590c1552c0 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.950+0000 7f5909ffb640 1 -- 172.21.15.44:0/4290742522 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f4031830 con 0x7f590c1552c0 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.950+0000 7f5909ffb640 1 -- 172.21.15.44:0/4290742522 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4031be0 con 0x7f590c1552c0 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.951+0000 7f5911a3e640 1 -- 172.21.15.44:0/4290742522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c1552c0 msgr2=0x7f590c0b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.951+0000 7f5911a3e640 1 --2- 172.21.15.44:0/4290742522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c1552c0 0x7f590c0b32a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f58f40097a0 tx=0x7f58f402b8d0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.951+0000 7f5911a3e640 1 -- 172.21.15.44:0/4290742522 shutdown_connections 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.951+0000 7f5911a3e640 1 --2- 172.21.15.44:0/4290742522 >> v2:172.21.15.70:3300/0 conn(0x7f590c0caec0 0x7f590c0d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.951+0000 7f5911a3e640 1 --2- 172.21.15.44:0/4290742522 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c1552c0 0x7f590c0b32a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.951+0000 7f5911a3e640 1 -- 172.21.15.44:0/4290742522 >> 172.21.15.44:0/4290742522 conn(0x7f590c0676b0 msgr2=0x7f590c067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.951+0000 7f5911a3e640 1 -- 172.21.15.44:0/4290742522 shutdown_connections 2024-04-22T07:05:35.949 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.951+0000 7f5911a3e640 1 -- 172.21.15.44:0/4290742522 wait complete. 2024-04-22T07:05:35.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.952+0000 7f5911a3e640 1 Processor -- start 2024-04-22T07:05:35.950 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.952+0000 7f5911a3e640 1 -- start start 2024-04-22T07:05:35.951 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f5911a3e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c0caec0 0x7f590c0c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f5911a3e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f590c0cc920 0x7f590c14de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f5911a3e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f590c1552c0 0x7f590c14e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f5911a3e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f590c0c6a00 con 0x7f590c1552c0 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f5911a3e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f590c15bce0 con 0x7f590c0cc920 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f5911a3e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f590c0cda20 con 0x7f590c0caec0 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f590c1552c0 0x7f590c14e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f590c1552c0 0x7f590c14e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34394/0 (socket says 172.21.15.44:34394) 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590b7fe640 1 -- 172.21.15.44:0/1127264593 learned_addr learned my addr 172.21.15.44:0/1127264593 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590a7fc640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f590c0cc920 0x7f590c14de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590affd640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c0caec0 0x7f590c0c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590b7fe640 1 -- 172.21.15.44:0/1127264593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c0caec0 msgr2=0x7f590c0c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590b7fe640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c0caec0 0x7f590c0c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590b7fe640 1 -- 172.21.15.44:0/1127264593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f590c0cc920 msgr2=0x7f590c14de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.952 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590b7fe640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f590c0cc920 0x7f590c14de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590b7fe640 1 -- 172.21.15.44:0/1127264593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590c14f190 con 0x7f590c1552c0 2024-04-22T07:05:35.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.953+0000 7f590b7fe640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f590c1552c0 0x7f590c14e3d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f590400cb70 tx=0x7f590400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.954+0000 7f5903fff640 1 -- 172.21.15.44:0/1127264593 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904017070 con 0x7f590c1552c0 2024-04-22T07:05:35.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.954+0000 7f5903fff640 1 -- 172.21.15.44:0/1127264593 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f590400ad40 con 0x7f590c1552c0 2024-04-22T07:05:35.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.954+0000 7f5911a3e640 1 -- 172.21.15.44:0/1127264593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590c1e5ef0 con 0x7f590c1552c0 2024-04-22T07:05:35.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.954+0000 7f5903fff640 1 -- 172.21.15.44:0/1127264593 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904012620 con 0x7f590c1552c0 2024-04-22T07:05:35.953 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.954+0000 7f5911a3e640 1 -- 172.21.15.44:0/1127264593 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590c1e6440 con 0x7f590c1552c0 2024-04-22T07:05:35.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.955+0000 7f5903fff640 1 -- 172.21.15.44:0/1127264593 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f590401a070 con 0x7f590c1552c0 2024-04-22T07:05:35.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.956+0000 7f5903fff640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f58ec06c8f0 0x7f58ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.956+0000 7f5903fff640 1 -- 172.21.15.44:0/1127264593 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f590408b9f0 con 0x7f590c1552c0 2024-04-22T07:05:35.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.956+0000 7f590affd640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f58ec06c8f0 0x7f58ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.956+0000 7f591462a740 1 Processor -- start 2024-04-22T07:05:35.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.956+0000 7f590affd640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f58ec06c8f0 0x7f58ec06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f58f4036000 tx=0x7f58f403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.954 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.957+0000 7f591462a740 1 -- start start 2024-04-22T07:05:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.957+0000 7f591462a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c437d0 0x55da24c43bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.957+0000 7f591462a740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c2d2d0 0x55da24c44900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.957+0000 7f591462a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c02810 0x55da24c44e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.957+0000 7f591462a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55da24b69500 con 0x55da24c02810 2024-04-22T07:05:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.957+0000 7f591462a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55da24b0f6a0 con 0x55da24c437d0 2024-04-22T07:05:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.957+0000 7f591462a740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55da24b654c0 con 0x55da24c2d2d0 2024-04-22T07:05:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c02810 0x55da24c44e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.956 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c02810 0x55da24c44e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34396/0 (socket says 172.21.15.44:34396) 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c437d0 0x55da24c43bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590b7fe640 1 -- 172.21.15.44:0/1646275742 learned_addr learned my addr 172.21.15.44:0/1646275742 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590a7fc640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c2d2d0 0x55da24c44900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590b7fe640 1 -- 172.21.15.44:0/1646275742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c2d2d0 msgr2=0x55da24c44900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590b7fe640 1 --2- 172.21.15.44:0/1646275742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c2d2d0 0x55da24c44900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590b7fe640 1 -- 172.21.15.44:0/1646275742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c437d0 msgr2=0x55da24c43bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590b7fe640 1 --2- 172.21.15.44:0/1646275742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c437d0 0x55da24c43bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590b7fe640 1 -- 172.21.15.44:0/1646275742 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da24b2e5e0 con 0x55da24c02810 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f590b7fe640 1 --2- 172.21.15.44:0/1646275742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c02810 0x55da24c44e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5904089ea0 tx=0x7f590405d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99685df64042b5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f59017fa640 1 -- 172.21.15.44:0/1646275742 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59040579d0 con 0x55da24c02810 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f59017fa640 1 -- 172.21.15.44:0/1646275742 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5904055d40 con 0x55da24c02810 2024-04-22T07:05:35.957 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.958+0000 7f59017fa640 1 -- 172.21.15.44:0/1646275742 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904061f70 con 0x55da24c02810 2024-04-22T07:05:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.959+0000 7f591462a740 1 -- 172.21.15.44:0/1646275742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c02810 msgr2=0x55da24c44e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.959+0000 7f591462a740 1 --2- 172.21.15.44:0/1646275742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c02810 0x55da24c44e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5904089ea0 tx=0x7f590405d9d0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.959+0000 7f591462a740 1 -- 172.21.15.44:0/1646275742 shutdown_connections 2024-04-22T07:05:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.959+0000 7f591462a740 1 --2- 172.21.15.44:0/1646275742 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c2d2d0 0x55da24c44900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.959+0000 7f591462a740 1 --2- 172.21.15.44:0/1646275742 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c02810 0x55da24c44e40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.959+0000 7f591462a740 1 --2- 172.21.15.44:0/1646275742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c437d0 0x55da24c43bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.962 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.959+0000 7f591462a740 1 -- 172.21.15.44:0/1646275742 >> 172.21.15.44:0/1646275742 conn(0x55da24c00550 msgr2=0x55da24c02e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.959+0000 7f591462a740 1 -- 172.21.15.44:0/1646275742 shutdown_connections 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.959+0000 7f591462a740 1 -- 172.21.15.44:0/1646275742 wait complete. 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.960+0000 7f591462a740 1 -- start start 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.960+0000 7f591462a740 1 Processor -- start 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.960+0000 7f591462a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c24220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.960+0000 7f591462a740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c45980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.960+0000 7f591462a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 0x55da24c47500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.960+0000 7f591462a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x55da24c01c30 con 0x55da24c2d2d0 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.960+0000 7f591462a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55da24b22a60 con 0x55da24c48520 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.960+0000 7f591462a740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55da24c2d7a0 con 0x55da24c437d0 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f591462a740 10 client.0 _ll_register_callbacks cb 0x55da2487b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c24220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c24220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34402/0 (socket says 172.21.15.44:34402) 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590affd640 1 -- 172.21.15.44:0/2553275239 learned_addr learned my addr 172.21.15.44:0/2553275239 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:35.963 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590b7fe640 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 0x55da24c47500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590a7fc640 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c45980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590affd640 1 -- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 msgr2=0x55da24c45980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590affd640 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c45980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590affd640 1 -- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 msgr2=0x55da24c47500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590affd640 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 0x55da24c47500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590affd640 1 -- 172.21.15.44:0/2553275239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da2487b920 con 0x55da24c2d2d0 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f590affd640 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c24220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58f4034040 tx=0x7f58f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a489b02007a255 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f58ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2553275239 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4035040 con 0x55da24c2d2d0 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2553275239 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f58f4003c60 con 0x55da24c2d2d0 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.961+0000 7f591462a740 1 -- 172.21.15.44:0/2553275239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5904055d40 con 0x55da24c2d2d0 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2553275239 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f58f405b050 con 0x55da24c2d2d0 2024-04-22T07:05:35.964 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f591462a740 10 client.5013 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f591462a740 1 -- 172.21.15.44:0/2553275239 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55da24947d30 con 0x55da24c2d2d0 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2553275239 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5904055d40 con 0x55da24c2d2d0 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2553275239 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f58f405e020 con 0x55da24c2d2d0 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f591462a740 10 client.5013 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f591462a740 10 client.5013 resolve_mds: validated gid 4324 aka daemon mds.f 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f591462a740 20 client.5013 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f591462a740 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55da24c54870 0x55da24bfe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f591462a740 4 client.5013 mds_command: new command op to 4324 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f591462a740 1 -- 172.21.15.44:0/2553275239 --> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da24b3ca10 con 0x55da24c54870 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.962+0000 7f590a7fc640 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55da24c54870 0x55da24bfe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f58ea7fc640 10 client.5013 ms_handle_connect on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f590a7fc640 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55da24c54870 0x55da24bfe790 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591a3835ac59439b server_cookie=c8bc595d68300bec in_seq=0 out_seq=0 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2553275239 <== mds.0 v2:172.21.15.70:6836/2872131879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da24b3ca10 con 0x55da24c54870 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f58ea7fc640 10 client.5013 handle_command_reply: tid=0 2024-04-22T07:05:35.965 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f591462a740 1 client.5013 shutdown 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f591462a740 1 -- 172.21.15.44:0/2553275239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 msgr2=0x55da24c24220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f591462a740 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c24220 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58f4034040 tx=0x7f58f4004730 comp rx=0 tx=0).stop 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f591462a740 1 -- 172.21.15.44:0/2553275239 shutdown_connections 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f591462a740 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c45980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f591462a740 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55da24c54870 0x55da24bfe790 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f591462a740 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 0x55da24c47500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f591462a740 1 --2- 172.21.15.44:0/2553275239 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c24220 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f58ea7fc640 0 client.5013 ms_handle_reset on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.963+0000 7f591462a740 1 -- 172.21.15.44:0/2553275239 >> 172.21.15.44:0/2553275239 conn(0x55da24c02810 msgr2=0x55da24c44870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.964+0000 7f591462a740 1 -- 172.21.15.44:0/2553275239 shutdown_connections 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.964+0000 7f591462a740 1 -- 172.21.15.44:0/2553275239 wait complete. 2024-04-22T07:05:35.966 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.964+0000 7f591462a740 20 client.5013 trim_cache size 0 max 16384 2024-04-22T07:05:35.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.981+0000 7f591462a740 1 Processor -- start 2024-04-22T07:05:35.979 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.981+0000 7f591462a740 1 -- start start 2024-04-22T07:05:35.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f591462a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c1f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f591462a740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c1f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.981 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f591462a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 0x55da24c471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f591462a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1fded0 con 0x55da24c2d2d0 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f591462a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1fc3e0 con 0x55da24c48520 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f591462a740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f590c1fd800 con 0x55da24c437d0 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c1f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590affd640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c1f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34410/0 (socket says 172.21.15.44:34410) 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590affd640 1 -- 172.21.15.44:0/2755469846 learned_addr learned my addr 172.21.15.44:0/2755469846 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590b7fe640 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 0x55da24c471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590a7fc640 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590affd640 1 -- 172.21.15.44:0/2755469846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 msgr2=0x55da24c1f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590affd640 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c1f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590affd640 1 -- 172.21.15.44:0/2755469846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 msgr2=0x55da24c471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590affd640 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 0x55da24c471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.982 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590affd640 1 -- 172.21.15.44:0/2755469846 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da24bffdc0 con 0x55da24c2d2d0 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590a7fc640 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c1f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.982+0000 7f590b7fe640 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 0x55da24c471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.983+0000 7f590affd640 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c1f0c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f58f402b910 tx=0x7f58f404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875c797e72e93075 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.983+0000 7f5900ff9640 1 -- 172.21.15.44:0/2755469846 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4003be0 con 0x55da24c2d2d0 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.983+0000 7f5900ff9640 1 -- 172.21.15.44:0/2755469846 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f4053070 con 0x55da24c2d2d0 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.983+0000 7f5900ff9640 1 -- 172.21.15.44:0/2755469846 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f403a780 con 0x55da24c2d2d0 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.983+0000 7f591462a740 1 -- 172.21.15.44:0/2755469846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 msgr2=0x55da24c1f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.983+0000 7f591462a740 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c1f0c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f58f402b910 tx=0x7f58f404c000 comp rx=0 tx=0).stop 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.984+0000 7f591462a740 1 -- 172.21.15.44:0/2755469846 shutdown_connections 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.984+0000 7f591462a740 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c1f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.984+0000 7f591462a740 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c48520 0x55da24c471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.984+0000 7f591462a740 1 --2- 172.21.15.44:0/2755469846 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c2d2d0 0x55da24c1f0c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.984+0000 7f591462a740 1 -- 172.21.15.44:0/2755469846 >> 172.21.15.44:0/2755469846 conn(0x55da24c02810 msgr2=0x55da24c24030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.984+0000 7f591462a740 1 -- 172.21.15.44:0/2755469846 shutdown_connections 2024-04-22T07:05:35.983 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.984+0000 7f591462a740 1 -- 172.21.15.44:0/2755469846 wait complete. 2024-04-22T07:05:35.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.984+0000 7f591462a740 1 -- start start 2024-04-22T07:05:35.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.984+0000 7f591462a740 1 Processor -- start 2024-04-22T07:05:35.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f591462a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c2d2d0 0x55da24c22270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f591462a740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c5e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f591462a740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c48520 0x55da24cb8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f591462a740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1fd130 con 0x55da24c48520 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f591462a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1fca60 con 0x55da24c2d2d0 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f591462a740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f590c1f3e20 con 0x55da24c437d0 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f591462a740 10 client.0 _ll_register_callbacks cb 0x7f58f4009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c48520 0x55da24cb8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590b7fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c48520 0x55da24cb8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:34416/0 (socket says 172.21.15.44:34416) 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590b7fe640 1 -- 172.21.15.44:0/2206869419 learned_addr learned my addr 172.21.15.44:0/2206869419 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590affd640 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c2d2d0 0x55da24c22270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590a7fc640 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c5e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590b7fe640 1 -- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 msgr2=0x55da24c5e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590b7fe640 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c5e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590b7fe640 1 -- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c2d2d0 msgr2=0x55da24c22270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:05:35.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590b7fe640 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c2d2d0 0x55da24c22270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590b7fe640 1 -- 172.21.15.44:0/2206869419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da24c27d70 con 0x55da24c48520 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.985+0000 7f590a7fc640 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c5e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f590b7fe640 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c48520 0x55da24cb8220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f59040623c0 tx=0x7f5904001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2faf29386af5419 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f58ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2206869419 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904061d70 con 0x55da24c48520 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2206869419 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 5825+0+0 (secure 0 0 0) 0x7f5904071e00 con 0x55da24c48520 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f591462a740 1 -- 172.21.15.44:0/2206869419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f4053070 con 0x55da24c48520 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2206869419 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f5904061f10 con 0x55da24c48520 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f591462a740 10 client.5019 fetch_fsmap learned FSMap version 17 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f591462a740 1 -- 172.21.15.44:0/2206869419 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55da24cbf3e0 con 0x55da24c48520 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2206869419 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f4053070 con 0x55da24c48520 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2206869419 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 17) ==== 4784+0+0 (secure 0 0 0) 0x7f590405eb60 con 0x55da24c48520 2024-04-22T07:05:35.988 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f591462a740 10 client.5019 fetch_fsmap finished waiting for FSMap version 17 2024-04-22T07:05:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f591462a740 10 client.5019 resolve_mds: validated gid 4324 aka daemon mds.f 2024-04-22T07:05:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f591462a740 20 client.5019 populate_metadata read hostname 'smithi044' 2024-04-22T07:05:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f591462a740 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55da24c04450 0x55da24cb8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:05:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f591462a740 4 client.5019 mds_command: new command op to 4324 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d430eb8b", "release": true, "await": true}] 2024-04-22T07:05:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.986+0000 7f591462a740 1 -- 172.21.15.44:0/2206869419 --> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d430eb8b", "release": true, "await": true}) -- 0x55da24b76f20 con 0x55da24c04450 2024-04-22T07:05:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.987+0000 7f590affd640 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55da24c04450 0x55da24cb8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:05:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.987+0000 7f590affd640 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55da24c04450 0x55da24cb8760 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee671bd02d0d40cd server_cookie=a0ccc1ddc26714a8 in_seq=0 out_seq=0 2024-04-22T07:05:35.989 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.987+0000 7f58ea7fc640 10 client.5019 ms_handle_connect on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:35.989 INFO:tasks.ceph.mds.c.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7ff43a9df640 time 2024-04-22T07:05:35.988600+0000 2024-04-22T07:05:35.990 INFO:tasks.ceph.mds.c.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:35.990 INFO:tasks.ceph.mds.e.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fe835e18640 time 2024-04-22T07:05:35.988613+0000 2024-04-22T07:05:35.990 INFO:tasks.ceph.mds.e.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:35.990 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:35.990 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fe843981f5c] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fe84398211a] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x560572754fd2] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x560572755091] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56057270fa96] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56057270f9f6] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x5605728ad3cf] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5605728fdc1b] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fe843089c02] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fe84310ec40] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr:2024-04-22T07:05:35.988+0000 7fe835e18640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fe835e18640 time 2024-04-22T07:05:35.988613+0000 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fe843981f5c] 2024-04-22T07:05:35.991 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fe84398211a] 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x560572754fd2] 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x560572755091] 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56057270fa96] 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56057270f9f6] 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x5605728ad3cf] 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5605728fdc1b] 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fe843089c02] 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fe84310ec40] 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr:*** Caught signal (Aborted) ** 2024-04-22T07:05:35.992 INFO:tasks.ceph.mds.e.smithi044.stderr: in thread 7fe835e18640 thread_name:quiesce.agt 2024-04-22T07:05:35.992 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f58ea7fc640 1 -- 172.21.15.44:0/2206869419 <== mds.0 v2:172.21.15.70:6836/2872131879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+587 (crc 0 0 0) 0x55da24b76f20 con 0x55da24c04450 2024-04-22T07:05:35.993 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f58ea7fc640 10 client.5019 handle_command_reply: tid=0 2024-04-22T07:05:35.993 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:35.993 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff448581f5c] 2024-04-22T07:05:35.993 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7ff44858211a] 2024-04-22T07:05:35.993 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5611813d1fd2] 2024-04-22T07:05:35.993 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5611813d2091] 2024-04-22T07:05:35.993 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56118138ca96] 2024-04-22T07:05:35.993 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56118138c9f6] 2024-04-22T07:05:35.993 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x56118152a3cf] 2024-04-22T07:05:35.993 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x56118157ac1b] 2024-04-22T07:05:35.993 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7ff447c89c02] 2024-04-22T07:05:35.994 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7ff447d0ec40] 2024-04-22T07:05:35.994 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 client.5019 shutdown 2024-04-22T07:05:35.994 INFO:tasks.ceph.mds.c.smithi044.stderr:*** Caught signal (Aborted) ** 2024-04-22T07:05:35.994 INFO:tasks.ceph.mds.c.smithi044.stderr: in thread 7ff43a9df640 thread_name:quiesce.agt 2024-04-22T07:05:35.994 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-04-22T07:05:35.988+0000 7ff43a9df640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7ff43a9df640 time 2024-04-22T07:05:35.988600+0000 2024-04-22T07:05:35.994 INFO:tasks.ceph.mds.c.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:35.995 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:35.998 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:35.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff448581f5c] 2024-04-22T07:05:35.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7ff44858211a] 2024-04-22T07:05:35.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5611813d1fd2] 2024-04-22T07:05:35.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5611813d2091] 2024-04-22T07:05:35.998 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56118138ca96] 2024-04-22T07:05:35.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56118138c9f6] 2024-04-22T07:05:35.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x56118152a3cf] 2024-04-22T07:05:35.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x56118157ac1b] 2024-04-22T07:05:35.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7ff447c89c02] 2024-04-22T07:05:35.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7ff447d0ec40] 2024-04-22T07:05:35.999 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 -- 172.21.15.44:0/2206869419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c48520 msgr2=0x55da24cb8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c48520 0x55da24cb8220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f59040623c0 tx=0x7f5904001250 comp rx=0 tx=0).stop 2024-04-22T07:05:36.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 -- 172.21.15.44:0/2206869419 shutdown_connections 2024-04-22T07:05:36.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55da24c437d0 0x55da24c5e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55da24c04450 0x55da24cb8760 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x55da24c48520 0x55da24cb8220 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 --2- 172.21.15.44:0/2206869419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55da24c2d2d0 0x55da24c22270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f58ea7fc640 0 client.5019 ms_handle_reset on v2:172.21.15.70:6836/2872131879 2024-04-22T07:05:36.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 -- 172.21.15.44:0/2206869419 >> 172.21.15.44:0/2206869419 conn(0x55da24c02810 msgr2=0x55da24c1ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:36.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 -- 172.21.15.44:0/2206869419 shutdown_connections 2024-04-22T07:05:36.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.988+0000 7f591462a740 1 -- 172.21.15.44:0/2206869419 wait complete. 2024-04-22T07:05:36.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f591462a740 20 client.5019 trim_cache size 0 max 16384 2024-04-22T07:05:36.001 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fe84303e6f0] 2024-04-22T07:05:36.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fe84308b94c] 2024-04-22T07:05:36.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: raise() 2024-04-22T07:05:36.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: abort() 2024-04-22T07:05:36.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fe843981fb6] 2024-04-22T07:05:36.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fe84398211a] 2024-04-22T07:05:36.001 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x560572754fd2] 2024-04-22T07:05:36.002 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x560572755091] 2024-04-22T07:05:36.002 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56057270fa96] 2024-04-22T07:05:36.002 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56057270f9f6] 2024-04-22T07:05:36.002 INFO:tasks.ceph.mds.e.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x5605728ad3cf] 2024-04-22T07:05:36.002 INFO:tasks.ceph.mds.e.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5605728fdc1b] 2024-04-22T07:05:36.002 INFO:tasks.ceph.mds.e.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fe843089c02] 2024-04-22T07:05:36.002 INFO:tasks.ceph.mds.e.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fe84310ec40] 2024-04-22T07:05:36.002 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:05:36.002 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 17, 2024-04-22T07:05:36.002 INFO:teuthology.orchestra.run.smithi044.stdout: "leader": 4324, 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "set_version": 4, 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "sets": { 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "d430eb8b": { 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "version": 4, 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "age_ref": 0.0, 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RELEASED", 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "timeout": 60.0, 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "expiration": 79.4, 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "members": { 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "file:/": { 2024-04-22T07:05:36.003 INFO:teuthology.orchestra.run.smithi044.stdout: "excluded": false, 2024-04-22T07:05:36.004 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T07:05:36.004 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "RELEASED", 2024-04-22T07:05:36.004 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T07:05:36.004 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:05:36.004 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:05:36.004 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:05:36.004 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:05:36.004 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:05:36.004 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:05:36.005 INFO:tasks.ceph.mds.b.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fdaee05b640 time 2024-04-22T07:05:35.988634+0000 2024-04-22T07:05:36.005 INFO:tasks.ceph.mds.b.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.005 INFO:tasks.ceph.mds.e.smithi044.stderr:2024-04-22T07:05:35.989+0000 7fe835e18640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.005 INFO:tasks.ceph.mds.e.smithi044.stderr: in thread 7fe835e18640 thread_name:quiesce.agt 2024-04-22T07:05:36.005 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:36.005 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.005 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fe84303e6f0] 2024-04-22T07:05:36.005 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fe84308b94c] 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: raise() 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: abort() 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fe843981fb6] 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fe84398211a] 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x560572754fd2] 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x560572755091] 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56057270fa96] 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56057270f9f6] 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x5605728ad3cf] 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5605728fdc1b] 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fe843089c02] 2024-04-22T07:05:36.006 INFO:tasks.ceph.mds.e.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fe84310ec40] 2024-04-22T07:05:36.007 INFO:tasks.ceph.mds.e.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.007 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:36.007 INFO:tasks.ceph.mds.d.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f30747b2640 time 2024-04-22T07:05:35.988616+0000 2024-04-22T07:05:36.007 INFO:tasks.ceph.mds.d.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.007 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.007 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff447c3e6f0] 2024-04-22T07:05:36.007 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7ff447c8b94c] 2024-04-22T07:05:36.007 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: raise() 2024-04-22T07:05:36.007 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: abort() 2024-04-22T07:05:36.007 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7ff448581fb6] 2024-04-22T07:05:36.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7ff44858211a] 2024-04-22T07:05:36.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5611813d1fd2] 2024-04-22T07:05:36.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5611813d2091] 2024-04-22T07:05:36.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56118138ca96] 2024-04-22T07:05:36.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56118138c9f6] 2024-04-22T07:05:36.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x56118152a3cf] 2024-04-22T07:05:36.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x56118157ac1b] 2024-04-22T07:05:36.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7ff447c89c02] 2024-04-22T07:05:36.008 INFO:tasks.ceph.mds.c.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7ff447d0ec40] 2024-04-22T07:05:36.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 -- 172.21.15.44:0/1127264593 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f58ec06c8f0 msgr2=0x7f58ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f58ec06c8f0 0x7f58ec06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f58f4036000 tx=0x7f58f403c040 comp rx=0 tx=0).stop 2024-04-22T07:05:36.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 -- 172.21.15.44:0/1127264593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f590c1552c0 msgr2=0x7f590c14e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:05:36.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f590c1552c0 0x7f590c14e3d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f590400cb70 tx=0x7f590400a430 comp rx=0 tx=0).stop 2024-04-22T07:05:36.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 -- 172.21.15.44:0/1127264593 shutdown_connections 2024-04-22T07:05:36.009 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.009 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fdafbb81f5c] 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fdafbb8211a] 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55cfabd94fd2] 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55cfabd95091] 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55cfabd4fa96] 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55cfabd4f9f6] 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x55cfabeed3cf] 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55cfabf3dc1b] 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fdafb289c02] 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fdafb30ec40] 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr:2024-04-22T07:05:35.987+0000 7fdaee05b640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fdaee05b640 time 2024-04-22T07:05:35.988634+0000 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.010 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fdafbb81f5c] 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fdafbb8211a] 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55cfabd94fd2] 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55cfabd95091] 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55cfabd4fa96] 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55cfabd4f9f6] 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x55cfabeed3cf] 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55cfabf3dc1b] 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fdafb289c02] 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fdafb30ec40] 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr:*** Caught signal (Aborted) ** 2024-04-22T07:05:36.011 INFO:tasks.ceph.mds.b.smithi070.stderr: in thread 7fdaee05b640 thread_name:quiesce.agt 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr:2024-04-22T07:05:35.989+0000 7ff43a9df640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: in thread 7ff43a9df640 thread_name:quiesce.agt 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff447c3e6f0] 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7ff447c8b94c] 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: raise() 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: abort() 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7ff448581fb6] 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7ff44858211a] 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5611813d1fd2] 2024-04-22T07:05:36.012 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5611813d2091] 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56118138ca96] 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56118138c9f6] 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.c.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x56118152a3cf] 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.c.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x56118157ac1b] 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.c.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7ff447c89c02] 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.c.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7ff447d0ec40] 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.c.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.d.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.d.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f3082381f5c] 2024-04-22T07:05:36.013 INFO:tasks.ceph.mds.d.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f308238211a] 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x557b59271fd2] 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x557b59272091] 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x557b5922ca96] 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x557b5922c9f6] 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x557b593ca3cf] 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x557b5941ac1b] 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f3081a89c02] 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f3081b0ec40] 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr:*** Caught signal (Aborted) ** 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr: in thread 7f30747b2640 thread_name:quiesce.agt 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-04-22T07:05:35.987+0000 7f30747b2640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f30747b2640 time 2024-04-22T07:05:35.988616+0000 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.014 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f3082381f5c] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f308238211a] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x557b59271fd2] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x557b59272091] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x557b5922ca96] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x557b5922c9f6] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x557b593ca3cf] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x557b5941ac1b] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f3081a89c02] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f3081b0ec40] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3081a3e6f0] 2024-04-22T07:05:36.015 INFO:tasks.ceph.mds.d.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f3081a8b94c] 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 3: raise() 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 4: abort() 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f3082381fb6] 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f308238211a] 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x557b59271fd2] 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x557b59272091] 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x557b5922ca96] 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x557b5922c9f6] 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x557b593ca3cf] 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x557b5941ac1b] 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f3081a89c02] 2024-04-22T07:05:36.016 INFO:tasks.ceph.mds.d.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f3081b0ec40] 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fdafb23e6f0] 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fdafb28b94c] 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: raise() 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: abort() 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fdafbb81fb6] 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fdafbb8211a] 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55cfabd94fd2] 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55cfabd95091] 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55cfabd4fa96] 2024-04-22T07:05:36.017 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55cfabd4f9f6] 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x55cfabeed3cf] 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55cfabf3dc1b] 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fdafb289c02] 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fdafb30ec40] 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr:2024-04-22T07:05:35.988+0000 7fdaee05b640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: in thread 7fdaee05b640 thread_name:quiesce.agt 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fdafb23e6f0] 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fdafb28b94c] 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: raise() 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: abort() 2024-04-22T07:05:36.018 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fdafbb81fb6] 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fdafbb8211a] 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55cfabd94fd2] 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55cfabd95091] 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55cfabd4fa96] 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55cfabd4f9f6] 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x55cfabeed3cf] 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55cfabf3dc1b] 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fdafb289c02] 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fdafb30ec40] 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.019 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f58ec06c8f0 0x7f58ec06edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f590c1552c0 0x7f590c14e3d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f590c0cc920 0x7f590c14de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 --2- 172.21.15.44:0/1127264593 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f590c0caec0 0x7f590c0c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:05:36.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 -- 172.21.15.44:0/1127264593 >> 172.21.15.44:0/1127264593 conn(0x7f590c0676b0 msgr2=0x7f590c0ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:05:36.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 -- 172.21.15.44:0/1127264593 shutdown_connections 2024-04-22T07:05:36.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:05:35.989+0000 7f5911a3e640 1 -- 172.21.15.44:0/1127264593 wait complete. 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr:2024-04-22T07:05:35.988+0000 7f30747b2640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: in thread 7f30747b2640 thread_name:quiesce.agt 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3081a3e6f0] 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f3081a8b94c] 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: 3: raise() 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: 4: abort() 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f3082381fb6] 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f308238211a] 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x557b59271fd2] 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x557b59272091] 2024-04-22T07:05:36.021 INFO:tasks.ceph.mds.d.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x557b5922ca96] 2024-04-22T07:05:36.022 INFO:tasks.ceph.mds.d.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x557b5922c9f6] 2024-04-22T07:05:36.022 INFO:tasks.ceph.mds.d.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x557b593ca3cf] 2024-04-22T07:05:36.022 INFO:tasks.ceph.mds.d.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x557b5941ac1b] 2024-04-22T07:05:36.022 INFO:tasks.ceph.mds.d.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f3081a89c02] 2024-04-22T07:05:36.022 INFO:tasks.ceph.mds.d.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f3081b0ec40] 2024-04-22T07:05:36.022 INFO:tasks.ceph.mds.d.smithi070.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.022 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.022 INFO:tasks.ceph.mds.d.smithi070.stderr: -1> 2024-04-22T07:05:35.987+0000 7f30747b2640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f30747b2640 time 2024-04-22T07:05:35.988616+0000 2024-04-22T07:05:36.022 INFO:tasks.ceph.mds.d.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.022 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f3082381f5c] 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f308238211a] 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x557b59271fd2] 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x557b59272091] 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x557b5922ca96] 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x557b5922c9f6] 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x557b593ca3cf] 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x557b5941ac1b] 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f3081a89c02] 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f3081b0ec40] 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.023 INFO:tasks.ceph.mds.d.smithi070.stderr: 0> 2024-04-22T07:05:35.988+0000 7f30747b2640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: in thread 7f30747b2640 thread_name:quiesce.agt 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3081a3e6f0] 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f3081a8b94c] 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 3: raise() 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 4: abort() 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f3082381fb6] 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f308238211a] 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x557b59271fd2] 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x557b59272091] 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x557b5922ca96] 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x557b5922c9f6] 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x557b593ca3cf] 2024-04-22T07:05:36.024 INFO:tasks.ceph.mds.d.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x557b5941ac1b] 2024-04-22T07:05:36.025 INFO:tasks.ceph.mds.d.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f3081a89c02] 2024-04-22T07:05:36.025 INFO:tasks.ceph.mds.d.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f3081b0ec40] 2024-04-22T07:05:36.025 INFO:tasks.ceph.mds.d.smithi070.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.025 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.025 INFO:tasks.ceph.mds.e.smithi044.stderr: -1> 2024-04-22T07:05:35.988+0000 7fe835e18640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fe835e18640 time 2024-04-22T07:05:35.988613+0000 2024-04-22T07:05:36.025 INFO:tasks.ceph.mds.e.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.025 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:36.025 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.025 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fe843981f5c] 2024-04-22T07:05:36.025 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fe84398211a] 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x560572754fd2] 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x560572755091] 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56057270fa96] 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56057270f9f6] 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x5605728ad3cf] 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5605728fdc1b] 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fe843089c02] 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fe84310ec40] 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 0> 2024-04-22T07:05:35.989+0000 7fe835e18640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: in thread 7fe835e18640 thread_name:quiesce.agt 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.026 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fe84303e6f0] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fe84308b94c] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: raise() 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: abort() 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fe843981fb6] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fe84398211a] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x560572754fd2] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x560572755091] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56057270fa96] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56057270f9f6] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x5605728ad3cf] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5605728fdc1b] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fe843089c02] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fe84310ec40] 2024-04-22T07:05:36.027 INFO:tasks.ceph.mds.e.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr: -1> 2024-04-22T07:05:35.987+0000 7fdaee05b640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fdaee05b640 time 2024-04-22T07:05:35.988634+0000 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fdafbb81f5c] 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fdafbb8211a] 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55cfabd94fd2] 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55cfabd95091] 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55cfabd4fa96] 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55cfabd4f9f6] 2024-04-22T07:05:36.028 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x55cfabeed3cf] 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55cfabf3dc1b] 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fdafb289c02] 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fdafb30ec40] 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 0> 2024-04-22T07:05:35.988+0000 7fdaee05b640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: in thread 7fdaee05b640 thread_name:quiesce.agt 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fdafb23e6f0] 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fdafb28b94c] 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: raise() 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: abort() 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fdafbb81fb6] 2024-04-22T07:05:36.029 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fdafbb8211a] 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55cfabd94fd2] 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55cfabd95091] 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55cfabd4fa96] 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55cfabd4f9f6] 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.b.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x55cfabeed3cf] 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.b.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55cfabf3dc1b] 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.b.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fdafb289c02] 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.b.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fdafb30ec40] 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.b.smithi070.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.c.smithi044.stderr: -1> 2024-04-22T07:05:35.988+0000 7ff43a9df640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7ff43a9df640 time 2024-04-22T07:05:35.988600+0000 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.c.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.030 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff448581f5c] 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7ff44858211a] 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5611813d1fd2] 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5611813d2091] 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56118138ca96] 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56118138c9f6] 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x56118152a3cf] 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x56118157ac1b] 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7ff447c89c02] 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7ff447d0ec40] 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: 0> 2024-04-22T07:05:35.989+0000 7ff43a9df640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.031 INFO:tasks.ceph.mds.c.smithi044.stderr: in thread 7ff43a9df640 thread_name:quiesce.agt 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff447c3e6f0] 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7ff447c8b94c] 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: raise() 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: abort() 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7ff448581fb6] 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7ff44858211a] 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5611813d1fd2] 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5611813d2091] 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56118138ca96] 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56118138c9f6] 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x56118152a3cf] 2024-04-22T07:05:36.032 INFO:tasks.ceph.mds.c.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x56118157ac1b] 2024-04-22T07:05:36.033 INFO:tasks.ceph.mds.c.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7ff447c89c02] 2024-04-22T07:05:36.033 INFO:tasks.ceph.mds.c.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7ff447d0ec40] 2024-04-22T07:05:36.033 INFO:tasks.ceph.mds.c.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.033 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.033 INFO:tasks.ceph.mds.d.smithi070.stderr: -1> 2024-04-22T07:05:35.987+0000 7f30747b2640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7f30747b2640 time 2024-04-22T07:05:35.988616+0000 2024-04-22T07:05:36.033 INFO:tasks.ceph.mds.d.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.033 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.033 INFO:tasks.ceph.mds.d.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.033 INFO:tasks.ceph.mds.d.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f3082381f5c] 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f308238211a] 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x557b59271fd2] 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x557b59272091] 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x557b5922ca96] 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x557b5922c9f6] 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x557b593ca3cf] 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x557b5941ac1b] 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7f3081a89c02] 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7f3081b0ec40] 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 0> 2024-04-22T07:05:35.988+0000 7f30747b2640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: in thread 7f30747b2640 thread_name:quiesce.agt 2024-04-22T07:05:36.034 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3081a3e6f0] 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f3081a8b94c] 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 3: raise() 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 4: abort() 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f3082381fb6] 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f308238211a] 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x557b59271fd2] 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x557b59272091] 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x557b5922ca96] 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x557b5922c9f6] 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x557b593ca3cf] 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x557b5941ac1b] 2024-04-22T07:05:36.035 INFO:tasks.ceph.mds.d.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7f3081a89c02] 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.d.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7f3081b0ec40] 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.d.smithi070.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.d.smithi070.stderr: 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.e.smithi044.stderr: -1> 2024-04-22T07:05:35.988+0000 7fe835e18640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fe835e18640 time 2024-04-22T07:05:35.988613+0000 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.e.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fe843981f5c] 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fe84398211a] 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x560572754fd2] 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x560572755091] 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56057270fa96] 2024-04-22T07:05:36.036 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56057270f9f6] 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x5605728ad3cf] 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x5605728fdc1b] 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fe843089c02] 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fe84310ec40] 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 0> 2024-04-22T07:05:35.989+0000 7fe835e18640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: in thread 7fe835e18640 thread_name:quiesce.agt 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fe84303e6f0] 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fe84308b94c] 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 3: raise() 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 4: abort() 2024-04-22T07:05:36.037 INFO:tasks.ceph.mds.e.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fe843981fb6] 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fe84398211a] 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x560572754fd2] 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x560572755091] 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56057270fa96] 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56057270f9f6] 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x5605728ad3cf] 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x5605728fdc1b] 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fe843089c02] 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fe84310ec40] 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.e.smithi044.stderr: 2024-04-22T07:05:36.038 INFO:tasks.ceph.mds.b.smithi070.stderr: -1> 2024-04-22T07:05:35.987+0000 7fdaee05b640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7fdaee05b640 time 2024-04-22T07:05:35.988634+0000 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fdafbb81f5c] 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fdafbb8211a] 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55cfabd94fd2] 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55cfabd95091] 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55cfabd4fa96] 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55cfabd4f9f6] 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: ceph-mds(+0x48e3cf) [0x55cfabeed3cf] 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x55cfabf3dc1b] 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7fdafb289c02] 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7fdafb30ec40] 2024-04-22T07:05:36.039 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 0> 2024-04-22T07:05:35.988+0000 7fdaee05b640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: in thread 7fdaee05b640 thread_name:quiesce.agt 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fdafb23e6f0] 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fdafb28b94c] 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 3: raise() 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 4: abort() 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fdafbb81fb6] 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fdafbb8211a] 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x55cfabd94fd2] 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x55cfabd95091] 2024-04-22T07:05:36.040 INFO:tasks.ceph.mds.b.smithi070.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x55cfabd4fa96] 2024-04-22T07:05:36.041 INFO:tasks.ceph.mds.b.smithi070.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x55cfabd4f9f6] 2024-04-22T07:05:36.041 INFO:tasks.ceph.mds.b.smithi070.stderr: 11: ceph-mds(+0x48e3cf) [0x55cfabeed3cf] 2024-04-22T07:05:36.041 INFO:tasks.ceph.mds.b.smithi070.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x55cfabf3dc1b] 2024-04-22T07:05:36.041 INFO:tasks.ceph.mds.b.smithi070.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7fdafb289c02] 2024-04-22T07:05:36.041 INFO:tasks.ceph.mds.b.smithi070.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7fdafb30ec40] 2024-04-22T07:05:36.041 INFO:tasks.ceph.mds.b.smithi070.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.041 INFO:tasks.ceph.mds.b.smithi070.stderr: 2024-04-22T07:05:36.042 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: d430eb8b, seconds elapsed: 20.5 2024-04-22T07:05:36.042 INFO:tasks.ceph.mds.c.smithi044.stderr: -1> 2024-04-22T07:05:35.988+0000 7ff43a9df640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: In function 'void Locker::_drop_locks(MutationImpl*, std::set*, bool)' thread 7ff43a9df640 time 2024-04-22T07:05:35.988600+0000 2024-04-22T07:05:36.042 INFO:tasks.ceph.mds.c.smithi044.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3156-g86f7587a/rpm/el9/BUILD/ceph-19.0.0-3156-g86f7587a/src/mds/Locker.cc: 720: FAILED ceph_assert(lock->get_sm()->can_remote_xlock) 2024-04-22T07:05:36.042 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.042 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.042 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7ff448581f5c] 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7ff44858211a] 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5611813d1fd2] 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5611813d2091] 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56118138ca96] 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56118138c9f6] 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: ceph-mds(+0x48e3cf) [0x56118152a3cf] 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (QuiesceAgent::agent_thread_main()+0x129b) [0x56118157ac1b] 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: /lib64/libc.so.6(+0x89c02) [0x7ff447c89c02] 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: /lib64/libc.so.6(+0x10ec40) [0x7ff447d0ec40] 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.043 INFO:tasks.ceph.mds.c.smithi044.stderr: 0> 2024-04-22T07:05:35.989+0000 7ff43a9df640 -1 *** Caught signal (Aborted) ** 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: in thread 7ff43a9df640 thread_name:quiesce.agt 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: ceph version 19.0.0-3156-g86f7587a (86f7587a5a09af35f0895e1a2d08527638fae697) squid (dev) 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7ff447c3e6f0] 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7ff447c8b94c] 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 3: raise() 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 4: abort() 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7ff448581fb6] 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7ff44858211a] 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 7: (Locker::_drop_locks(MutationImpl*, std::set, std::allocator >*, bool)+0xac2) [0x5611813d1fd2] 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 8: (Locker::drop_locks(MutationImpl*, std::set, std::allocator >*)+0x81) [0x5611813d2091] 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 9: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x246) [0x56118138ca96] 2024-04-22T07:05:36.044 INFO:tasks.ceph.mds.c.smithi044.stderr: 10: (MDCache::request_cleanup(boost::intrusive_ptr const&)+0x1a6) [0x56118138c9f6] 2024-04-22T07:05:36.045 INFO:tasks.ceph.mds.c.smithi044.stderr: 11: ceph-mds(+0x48e3cf) [0x56118152a3cf] 2024-04-22T07:05:36.045 INFO:tasks.ceph.mds.c.smithi044.stderr: 12: (QuiesceAgent::agent_thread_main()+0x129b) [0x56118157ac1b] 2024-04-22T07:05:36.045 INFO:tasks.ceph.mds.c.smithi044.stderr: 13: /lib64/libc.so.6(+0x89c02) [0x7ff447c89c02] 2024-04-22T07:05:36.045 INFO:tasks.ceph.mds.c.smithi044.stderr: 14: /lib64/libc.so.6(+0x10ec40) [0x7ff447d0ec40] 2024-04-22T07:05:36.045 INFO:tasks.ceph.mds.c.smithi044.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T07:05:36.045 INFO:tasks.ceph.mds.c.smithi044.stderr: 2024-04-22T07:05:36.424 INFO:tasks.ceph.mds.c.smithi044.stderr:daemon-helper: command crashed with signal 6 2024-04-22T07:05:36.424 INFO:tasks.ceph.mds.e.smithi044.stderr:daemon-helper: command crashed with signal 6 2024-04-22T07:05:36.432 INFO:tasks.ceph.mds.b.smithi070.stderr:daemon-helper: command crashed with signal 6 2024-04-22T07:05:36.433 INFO:tasks.ceph.mds.d.smithi070.stderr:daemon-helper: command crashed with signal 6 2024-04-22T07:05:37.732 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-22T07:05:37.732 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2024-04-22T07:05:37.732 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-22T07:05:37.732 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-22T07:05:40.747 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:05:40.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:05:44.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~7s 2024-04-22T07:05:44.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~7s 2024-04-22T07:05:44.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2024-04-22T07:05:44.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2024-04-22T07:05:50.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~13s 2024-04-22T07:05:50.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~13s 2024-04-22T07:05:50.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2024-04-22T07:05:50.759 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~13s 2024-04-22T07:05:57.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~20s 2024-04-22T07:05:57.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~20s 2024-04-22T07:05:57.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~20s 2024-04-22T07:05:57.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~20s 2024-04-22T07:06:03.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~26s 2024-04-22T07:06:03.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~26s 2024-04-22T07:06:03.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~26s 2024-04-22T07:06:03.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~26s 2024-04-22T07:06:10.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~33s 2024-04-22T07:06:10.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~33s 2024-04-22T07:06:10.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~33s 2024-04-22T07:06:10.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~33s 2024-04-22T07:06:10.850 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:06:10.853 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:06:12.071 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-22T07:06:12.071 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.6 2024-04-22T07:06:12.072 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:06:12.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.176+0000 7fe25063f640 1 Processor -- start 2024-04-22T07:06:12.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.176+0000 7fe25063f640 1 -- start start 2024-04-22T07:06:12.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.177+0000 7fe25063f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fe24815efc0 0x7fe2480c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.177+0000 7fe25063f640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe248155500 con 0x7fe248155120 2024-04-22T07:06:12.176 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.177+0000 7fe25063f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe248072f90 con 0x7fe2480caec0 2024-04-22T07:06:12.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.177+0000 7fe25063f640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fe24815b0d0 con 0x7fe24815efc0 2024-04-22T07:06:12.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.177+0000 7fe24e3db640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7fe248155120 0x7fe2480d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:43758/0 (socket says 172.21.15.44:43758) 2024-04-22T07:06:12.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.177+0000 7fe24e3db640 1 -- 172.21.15.44:0/1025566929 learned_addr learned my addr 172.21.15.44:0/1025566929 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:06:12.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.177+0000 7fe24ebdc640 1 --2- 172.21.15.44:0/1025566929 >> v2:172.21.15.44:3300/0 conn(0x7fe24815efc0 0x7fe2480c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.178+0000 7fe24ebdc640 1 -- 172.21.15.44:0/1025566929 >> v1:172.21.15.70:6789/0 conn(0x7fe2480caec0 legacy=0x7fe24815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.178+0000 7fe24ebdc640 1 -- 172.21.15.44:0/1025566929 >> v1:172.21.15.44:6789/0 conn(0x7fe248155120 legacy=0x7fe2480d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.177 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.178+0000 7fe24ebdc640 1 -- 172.21.15.44:0/1025566929 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24815b3b0 con 0x7fe24815efc0 2024-04-22T07:06:12.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.179+0000 7fe24ebdc640 1 --2- 172.21.15.44:0/1025566929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe24815efc0 0x7fe2480c6380 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe240009d90 tx=0x7fe24002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f356c0a13baa1eed server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.180+0000 7fe24d3d9640 1 -- 172.21.15.44:0/1025566929 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240039070 con 0x7fe24815efc0 2024-04-22T07:06:12.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.180+0000 7fe24d3d9640 1 -- 172.21.15.44:0/1025566929 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24002ccf0 con 0x7fe24815efc0 2024-04-22T07:06:12.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.180+0000 7fe24d3d9640 1 -- 172.21.15.44:0/1025566929 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240034660 con 0x7fe24815efc0 2024-04-22T07:06:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.180+0000 7fe25063f640 1 -- 172.21.15.44:0/1025566929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe24815efc0 msgr2=0x7fe2480c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.180+0000 7fe25063f640 1 --2- 172.21.15.44:0/1025566929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe24815efc0 0x7fe2480c6380 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe240009d90 tx=0x7fe24002bf40 comp rx=0 tx=0).stop 2024-04-22T07:06:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.181+0000 7fe25063f640 1 -- 172.21.15.44:0/1025566929 shutdown_connections 2024-04-22T07:06:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.181+0000 7fe25063f640 1 --2- 172.21.15.44:0/1025566929 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe24815efc0 0x7fe2480c6380 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.181+0000 7fe25063f640 1 -- 172.21.15.44:0/1025566929 >> 172.21.15.44:0/1025566929 conn(0x7fe2480676b0 msgr2=0x7fe248067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.181+0000 7fe25063f640 1 -- 172.21.15.44:0/1025566929 shutdown_connections 2024-04-22T07:06:12.180 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.181+0000 7fe25063f640 1 -- 172.21.15.44:0/1025566929 wait complete. 2024-04-22T07:06:12.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.182+0000 7fe25063f640 1 Processor -- start 2024-04-22T07:06:12.181 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.183+0000 7fe25063f640 1 -- start start 2024-04-22T07:06:12.183 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.183+0000 7fe25063f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2480caec0 0x7fe2481517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.183+0000 7fe25063f640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe248155120 0x7fe2481f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.183+0000 7fe25063f640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe24815efc0 0x7fe2481f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.183+0000 7fe25063f640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fe24815bc50 con 0x7fe248155120 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.183+0000 7fe25063f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe24815b970 con 0x7fe2480caec0 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.183+0000 7fe24dbda640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe248155120 0x7fe2481f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.183+0000 7fe25063f640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fe24815e390 con 0x7fe24815efc0 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.183+0000 7fe24dbda640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe248155120 0x7fe2481f0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58242/0 (socket says 172.21.15.44:58242) 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.183+0000 7fe24e3db640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2480caec0 0x7fe2481517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.184+0000 7fe24dbda640 1 -- 172.21.15.44:0/662294957 learned_addr learned my addr 172.21.15.44:0/662294957 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.184+0000 7fe24ebdc640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe24815efc0 0x7fe2481f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.184 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.184+0000 7fe24dbda640 1 -- 172.21.15.44:0/662294957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe24815efc0 msgr2=0x7fe2481f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.184+0000 7fe24dbda640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe24815efc0 0x7fe2481f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.184+0000 7fe24dbda640 1 -- 172.21.15.44:0/662294957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2480caec0 msgr2=0x7fe2481517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.184+0000 7fe24dbda640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2480caec0 0x7fe2481517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.184+0000 7fe24dbda640 1 -- 172.21.15.44:0/662294957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2481f43b0 con 0x7fe248155120 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.184+0000 7fe24ebdc640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe24815efc0 0x7fe2481f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.184+0000 7fe24dbda640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe248155120 0x7fe2481f0e60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe23800c460 tx=0x7fe23800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.184+0000 7fe24e3db640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2480caec0 0x7fe2481517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.185+0000 7fe23effd640 1 -- 172.21.15.44:0/662294957 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380052d0 con 0x7fe248155120 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.185+0000 7fe23effd640 1 -- 172.21.15.44:0/662294957 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe238005470 con 0x7fe248155120 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.185+0000 7fe23effd640 1 -- 172.21.15.44:0/662294957 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238012870 con 0x7fe248155120 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.185+0000 7fe25063f640 1 -- 172.21.15.44:0/662294957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2481f4690 con 0x7fe248155120 2024-04-22T07:06:12.185 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.185+0000 7fe25063f640 1 -- 172.21.15.44:0/662294957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2481f4c50 con 0x7fe248155120 2024-04-22T07:06:12.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.186+0000 7fe25063f640 1 -- 172.21.15.44:0/662294957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24804fdc0 con 0x7fe248155120 2024-04-22T07:06:12.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.187+0000 7fe23effd640 1 -- 172.21.15.44:0/662294957 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe238012a10 con 0x7fe248155120 2024-04-22T07:06:12.186 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.188+0000 7fe23effd640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe22c06c920 0x7fe22c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.188+0000 7fe23effd640 1 -- 172.21.15.44:0/662294957 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7fe23808b750 con 0x7fe248155120 2024-04-22T07:06:12.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.188+0000 7fe24e3db640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe22c06c920 0x7fe22c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.187 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.189+0000 7fe24e3db640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe22c06c920 0x7fe22c06ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe244000f60 tx=0x7fe244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.195 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.196+0000 7fe23effd640 1 -- 172.21.15.44:0/662294957 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe238059ab0 con 0x7fe248155120 2024-04-22T07:06:12.423 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.424+0000 7fe25063f640 1 -- 172.21.15.44:0/662294957 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2480ca970 con 0x7fe248155120 2024-04-22T07:06:12.424 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.425+0000 7fe23effd640 1 -- 172.21.15.44:0/662294957 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+6226 (secure 0 0 0) 0x7fe2380593b0 con 0x7fe248155120 2024-04-22T07:06:12.424 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:06:12.424 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-22T07:04:13.144503+0000","modified":"2024-04-22T07:05:54.237610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4324,"mds_1":4303,"mds_2":4300,"mds_3":4278,"mds_4":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"a","rank":4,"incarnation":18,"state":"up:resolve","state_seq":28,"addr":"172.21.15.44:6835/1185009943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6834","nonce":1185009943},{"type":"v1","addr":"172.21.15.44:6835","nonce":1185009943}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4278":{"gid":4278,"name":"b","rank":3,"incarnation":14,"state":"up:active","state_seq":10,"addr":"172.21.15.70:6835/4058009549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4058009549},{"type":"v1","addr":"172.21.15.70:6835","nonce":4058009549}]},"join_fscid":-1,"laggy_since":"2024-04-22T07:05:54.228756+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4300":{"gid":4300,"name":"c","rank":2,"incarnation":12,"state":"up:active","state_seq":8,"addr":"172.21.15.44:6838/1520421043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6836","nonce":1520421043},{"type":"v1","addr":"172.21.15.44:6838","nonce":1520421043}]},"join_fscid":-1,"laggy_since":"2024-04-22T07:05:49.227500+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4303":{"gid":4303,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":7,"addr":"172.21.15.44:6839/1911557586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6837","nonce":1911557586},{"type":"v1","addr":"172.21.15.44:6839","nonce":1911557586}]},"join_fscid":-1,"laggy_since":"2024-04-22T07:05:54.228778+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4324":{"gid":4324,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6837/2872131879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2872131879},{"type":"v1","addr":"172.21.15.70:6837","nonce":2872131879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4324,"qdb_cluster":[4278,4300,4303,4324]},"id":1}]} 2024-04-22T07:06:12.424 INFO:teuthology.orchestra.run.smithi044.stderr:dumped fsmap epoch 20 2024-04-22T07:06:12.426 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.427+0000 7fe25063f640 1 -- 172.21.15.44:0/662294957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe22c06c920 msgr2=0x7fe22c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.427+0000 7fe25063f640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe22c06c920 0x7fe22c06ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe244000f60 tx=0x7fe244009040 comp rx=0 tx=0).stop 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.427+0000 7fe25063f640 1 -- 172.21.15.44:0/662294957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe248155120 msgr2=0x7fe2481f0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.427+0000 7fe25063f640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe248155120 0x7fe2481f0e60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe23800c460 tx=0x7fe23800c920 comp rx=0 tx=0).stop 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.428+0000 7fe25063f640 1 -- 172.21.15.44:0/662294957 shutdown_connections 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.428+0000 7fe25063f640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fe22c06c920 0x7fe22c06ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.428+0000 7fe25063f640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fe24815efc0 0x7fe2481f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.428+0000 7fe25063f640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fe248155120 0x7fe2481f0e60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.428+0000 7fe25063f640 1 --2- 172.21.15.44:0/662294957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2480caec0 0x7fe2481517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.428+0000 7fe25063f640 1 -- 172.21.15.44:0/662294957 >> 172.21.15.44:0/662294957 conn(0x7fe2480676b0 msgr2=0x7fe2480c6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.428+0000 7fe25063f640 1 -- 172.21.15.44:0/662294957 shutdown_connections 2024-04-22T07:06:12.427 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.428+0000 7fe25063f640 1 -- 172.21.15.44:0/662294957 wait complete. 2024-04-22T07:06:12.438 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.4324 quiesce db / --timeout 60 --expiration 79.6 --await' 2024-04-22T07:06:12.438 DEBUG:teuthology.orchestra.run.smithi044:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4324 quiesce db / --timeout 60 --expiration 79.6 --await 2024-04-22T07:06:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.552+0000 7f524a9f4640 1 Processor -- start 2024-04-22T07:06:12.551 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.552+0000 7f524a9f4640 1 -- start start 2024-04-22T07:06:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f524a9f4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f52441562a0 0x7f52440cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f524a9f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52441595c0 0x7f52440caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.552 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f524a9f4640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52440c95b0 con 0x7f52440c8c50 2024-04-22T07:06:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f524a9f4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f5244159b20 con 0x7f52441562a0 2024-04-22T07:06:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f524a9f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f52440c9b30 con 0x7f52441595c0 2024-04-22T07:06:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f5243fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f52441562a0 0x7f52440cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f5243fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f52441562a0 0x7f52440cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58258/0 (socket says 172.21.15.44:58258) 2024-04-22T07:06:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f5243fff640 1 -- 172.21.15.44:0/3675794350 learned_addr learned my addr 172.21.15.44:0/3675794350 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f52437fe640 1 --2- 172.21.15.44:0/3675794350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52441595c0 0x7f52440caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f5243fff640 1 -- 172.21.15.44:0/3675794350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52441595c0 msgr2=0x7f52440caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f5243fff640 1 --2- 172.21.15.44:0/3675794350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52441595c0 0x7f52440caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.553+0000 7f5243fff640 1 -- 172.21.15.44:0/3675794350 >> v1:172.21.15.44:6789/0 conn(0x7f52440c8c50 legacy=0x7f52440c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.553 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.554+0000 7f5243fff640 1 -- 172.21.15.44:0/3675794350 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52440c9850 con 0x7f52441562a0 2024-04-22T07:06:12.554 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.555+0000 7f5243fff640 1 --2- 172.21.15.44:0/3675794350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52441562a0 0x7f52440cb890 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5234009870 tx=0x7f523402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2776b5ab2771be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.555+0000 7f5242ffd640 1 -- 172.21.15.44:0/3675794350 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523402f850 con 0x7f52441562a0 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.555+0000 7f5242ffd640 1 -- 172.21.15.44:0/3675794350 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523402f9f0 con 0x7f52441562a0 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.555+0000 7f5242ffd640 1 -- 172.21.15.44:0/3675794350 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523402fe00 con 0x7f52441562a0 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.556+0000 7f524a9f4640 1 -- 172.21.15.44:0/3675794350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52441562a0 msgr2=0x7f52440cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.556+0000 7f524a9f4640 1 --2- 172.21.15.44:0/3675794350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52441562a0 0x7f52440cb890 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5234009870 tx=0x7f523402ba30 comp rx=0 tx=0).stop 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.556+0000 7f524a9f4640 1 -- 172.21.15.44:0/3675794350 shutdown_connections 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.556+0000 7f524a9f4640 1 --2- 172.21.15.44:0/3675794350 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52441595c0 0x7f52440caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.556+0000 7f524a9f4640 1 --2- 172.21.15.44:0/3675794350 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52441562a0 0x7f52440cb890 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.556+0000 7f524a9f4640 1 -- 172.21.15.44:0/3675794350 >> 172.21.15.44:0/3675794350 conn(0x7f52440712d0 msgr2=0x7f52440716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.556+0000 7f524a9f4640 1 -- 172.21.15.44:0/3675794350 shutdown_connections 2024-04-22T07:06:12.555 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.556+0000 7f524a9f4640 1 -- 172.21.15.44:0/3675794350 wait complete. 2024-04-22T07:06:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.558+0000 7f524a9f4640 1 Processor -- start 2024-04-22T07:06:12.556 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.558+0000 7f524a9f4640 1 -- start start 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.559+0000 7f524a9f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52440c8c50 0x7f5244145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.559+0000 7f524a9f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52441562a0 0x7f5244147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.559+0000 7f524a9f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52441595c0 0x7f5244149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.559+0000 7f524a9f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f52440c4120 con 0x7f52440c8c50 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.559+0000 7f524a9f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f52440c34f0 con 0x7f52441562a0 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.559+0000 7f524a9f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f52440c42a0 con 0x7f52441595c0 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.559+0000 7f5243fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52440c8c50 0x7f5244145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.559+0000 7f5243fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52440c8c50 0x7f5244145350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58270/0 (socket says 172.21.15.44:58270) 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.559+0000 7f5243fff640 1 -- 172.21.15.44:0/3480643228 learned_addr learned my addr 172.21.15.44:0/3480643228 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.559+0000 7f52437fe640 1 --2- 172.21.15.44:0/3480643228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52441562a0 0x7f5244147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.560+0000 7f5243fff640 1 -- 172.21.15.44:0/3480643228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52441595c0 msgr2=0x7f5244149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.559 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.560+0000 7f5243fff640 1 --2- 172.21.15.44:0/3480643228 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f52441595c0 0x7f5244149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.560+0000 7f5243fff640 1 -- 172.21.15.44:0/3480643228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52441562a0 msgr2=0x7f5244147b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.560+0000 7f5243fff640 1 --2- 172.21.15.44:0/3480643228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52441562a0 0x7f5244147b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.560+0000 7f5243fff640 1 -- 172.21.15.44:0/3480643228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524414b410 con 0x7f52440c8c50 2024-04-22T07:06:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.560+0000 7f52437fe640 1 --2- 172.21.15.44:0/3480643228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52441562a0 0x7f5244147b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:06:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.560+0000 7f5243fff640 1 --2- 172.21.15.44:0/3480643228 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f52440c8c50 0x7f5244145350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5234000c00 tx=0x7f5234004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.560+0000 7f5240ff9640 1 -- 172.21.15.44:0/3480643228 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234037c40 con 0x7f52440c8c50 2024-04-22T07:06:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.560+0000 7f5240ff9640 1 -- 172.21.15.44:0/3480643228 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5234046070 con 0x7f52440c8c50 2024-04-22T07:06:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.560+0000 7f5240ff9640 1 -- 172.21.15.44:0/3480643228 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234041400 con 0x7f52440c8c50 2024-04-22T07:06:12.560 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.561+0000 7f524a9f4640 1 -- 172.21.15.44:0/3480643228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524414b6f0 con 0x7f52440c8c50 2024-04-22T07:06:12.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.561+0000 7f524a9f4640 1 -- 172.21.15.44:0/3480643228 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5244144b50 con 0x7f52440c8c50 2024-04-22T07:06:12.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.564+0000 7f524d5dd740 1 Processor -- start 2024-04-22T07:06:12.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.564+0000 7f524d5dd740 1 -- start start 2024-04-22T07:06:12.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f5240ff9640 1 -- 172.21.15.44:0/3480643228 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5234040070 con 0x7f52440c8c50 2024-04-22T07:06:12.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863177df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631c1890 0x555863178330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555863176470 0x5558631b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f524d5dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5558630d00b0 con 0x555863177a10 2024-04-22T07:06:12.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f524d5dd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x555863081680 con 0x555863176470 2024-04-22T07:06:12.571 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f524d5dd740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x55586308dc60 con 0x5558631c1890 2024-04-22T07:06:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f5240ff9640 1 --2- 172.21.15.44:0/3480643228 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f523006c8f0 0x7f523006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f5240ff9640 1 -- 172.21.15.44:0/3480643228 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f52340ae1a0 con 0x7f52440c8c50 2024-04-22T07:06:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f5248f91640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863177df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f5243fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631c1890 0x555863178330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f5243fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631c1890 0x555863178330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3301/0 says I am v2:172.21.15.44:42840/0 (socket says 172.21.15.44:42840) 2024-04-22T07:06:12.572 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f5243fff640 1 -- 172.21.15.44:0/1397651838 learned_addr learned my addr 172.21.15.44:0/1397651838 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f5248f91640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863177df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58276/0 (socket says 172.21.15.44:58276) 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f52437fe640 1 --2- 172.21.15.44:0/1397651838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555863176470 0x5558631b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.565+0000 7f5248f91640 1 --2- 172.21.15.44:0/3480643228 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f523006c8f0 0x7f523006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5243fff640 1 -- 172.21.15.44:0/1397651838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555863176470 msgr2=0x5558631b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5243fff640 1 --2- 172.21.15.44:0/1397651838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555863176470 0x5558631b7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5243fff640 1 -- 172.21.15.44:0/1397651838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 msgr2=0x555863177df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5243fff640 1 --2- 172.21.15.44:0/1397651838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863177df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5243fff640 1 -- 172.21.15.44:0/1397651838 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555862eb2c90 con 0x5558631c1890 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5248f91640 1 --2- 172.21.15.44:0/1397651838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863177df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5243fff640 1 --2- 172.21.15.44:0/1397651838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631c1890 0x555863178330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5234084540 tx=0x7f523402f950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e63ffabd4bf496 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5248f91640 1 --2- 172.21.15.44:0/3480643228 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f523006c8f0 0x7f523006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f523c002230 tx=0x7f523c001470 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5229ffb640 1 -- 172.21.15.44:0/1397651838 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523407be90 con 0x5558631c1890 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5229ffb640 1 -- 172.21.15.44:0/1397651838 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5234080240 con 0x5558631c1890 2024-04-22T07:06:12.573 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.566+0000 7f5229ffb640 1 -- 172.21.15.44:0/1397651838 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234084a70 con 0x5558631c1890 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.567+0000 7f524d5dd740 1 -- 172.21.15.44:0/1397651838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631c1890 msgr2=0x555863178330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.567+0000 7f524d5dd740 1 --2- 172.21.15.44:0/1397651838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631c1890 0x555863178330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5234084540 tx=0x7f523402f950 comp rx=0 tx=0).stop 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.567+0000 7f524d5dd740 1 -- 172.21.15.44:0/1397651838 shutdown_connections 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.567+0000 7f524d5dd740 1 --2- 172.21.15.44:0/1397651838 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631c1890 0x555863178330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.567+0000 7f524d5dd740 1 --2- 172.21.15.44:0/1397651838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x555863176470 0x5558631b7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.567+0000 7f524d5dd740 1 --2- 172.21.15.44:0/1397651838 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863177df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.567+0000 7f524d5dd740 1 -- 172.21.15.44:0/1397651838 >> 172.21.15.44:0/1397651838 conn(0x555863174410 msgr2=0x555863176ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.567+0000 7f524d5dd740 1 -- 172.21.15.44:0/1397651838 shutdown_connections 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.567+0000 7f524d5dd740 1 -- 172.21.15.44:0/1397651838 wait complete. 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.568+0000 7f524d5dd740 1 -- start start 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.568+0000 7f524d5dd740 1 Processor -- start 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631c1890 0x5558631c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.574 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 0x5558631cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5243fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f524d5dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x5558630d3400 con 0x555863177a10 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5243fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58280/0 (socket says 172.21.15.44:58280) 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5243fff640 1 -- 172.21.15.44:0/3940285576 learned_addr learned my addr 172.21.15.44:0/3940285576 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f524d5dd740 1 -- 172.21.15.44:0/3940285576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x55586308f840 con 0x5558631c1890 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f524d5dd740 1 -- 172.21.15.44:0/3940285576 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5558630d8720 con 0x5558631bfa90 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f524d5dd740 10 client.0 _ll_register_callbacks cb 0x5558630e0ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5248f91640 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 0x5558631cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f52437fe640 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631c1890 0x5558631c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5243fff640 1 -- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 msgr2=0x5558631cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5243fff640 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 0x5558631cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5243fff640 1 -- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631c1890 msgr2=0x5558631c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5243fff640 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631c1890 0x5558631c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.575 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5243fff640 1 -- 172.21.15.44:0/3940285576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555862f13420 con 0x555863177a10 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5243fff640 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631b5340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5234009870 tx=0x7f52340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a1812253c60950 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.570+0000 7f5222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f5222ffd640 1 -- 172.21.15.44:0/3940285576 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340847e0 con 0x555863177a10 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f5222ffd640 1 -- 172.21.15.44:0/3940285576 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f523409b220 con 0x555863177a10 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f524d5dd740 1 -- 172.21.15.44:0/3940285576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5234080240 con 0x555863177a10 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f5222ffd640 1 -- 172.21.15.44:0/3940285576 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5234084a70 con 0x555863177a10 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f524d5dd740 10 client.5043 fetch_fsmap learned FSMap version 20 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f524d5dd740 1 -- 172.21.15.44:0/3940285576 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x555862da6580 con 0x555863177a10 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f5222ffd640 1 -- 172.21.15.44:0/3940285576 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523409b070 con 0x555863177a10 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f5222ffd640 1 -- 172.21.15.44:0/3940285576 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 20) ==== 4171+0+0 (secure 0 0 0) 0x7f52340ac120 con 0x555863177a10 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f524d5dd740 10 client.5043 fetch_fsmap finished waiting for FSMap version 20 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f524d5dd740 10 client.5043 resolve_mds: validated gid 4324 aka daemon mds.f 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f524d5dd740 20 client.5043 populate_metadata read hostname 'smithi044' 2024-04-22T07:06:12.576 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f524d5dd740 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55586319b360 0x5558631b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f524d5dd740 4 client.5043 mds_command: new command op to 4324 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T07:06:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.571+0000 7f524d5dd740 1 -- 172.21.15.44:0/3940285576 --> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5234080240 con 0x55586319b360 2024-04-22T07:06:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.572+0000 7f52437fe640 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55586319b360 0x5558631b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.573+0000 7f5222ffd640 10 client.5043 ms_handle_connect on v2:172.21.15.70:6836/2872131879 2024-04-22T07:06:12.577 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.573+0000 7f52437fe640 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55586319b360 0x5558631b2790 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75bc742177e87732 server_cookie=186a97059a18d97 in_seq=0 out_seq=0 2024-04-22T07:06:12.578 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.573+0000 7f5222ffd640 1 -- 172.21.15.44:0/3940285576 <== mds.0 v2:172.21.15.70:6836/2872131879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f5234080240 con 0x55586319b360 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.573+0000 7f5222ffd640 10 client.5043 handle_command_reply: tid=0 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.573+0000 7f524d5dd740 1 client.5043 shutdown 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 1 -- 172.21.15.44:0/3940285576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 msgr2=0x5558631b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631b5340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5234009870 tx=0x7f52340046c0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 1 -- 172.21.15.44:0/3940285576 shutdown_connections 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 0x5558631cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x55586319b360 0x5558631b2790 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f5222ffd640 0 client.5043 ms_handle_reset on v2:172.21.15.70:6836/2872131879 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631c1890 0x5558631c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 1 --2- 172.21.15.44:0/3940285576 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631b5340 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 1 -- 172.21.15.44:0/3940285576 >> 172.21.15.44:0/3940285576 conn(0x555863176470 msgr2=0x555863168940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 1 -- 172.21.15.44:0/3940285576 shutdown_connections 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 1 -- 172.21.15.44:0/3940285576 wait complete. 2024-04-22T07:06:12.579 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.574+0000 7f524d5dd740 20 client.5043 trim_cache size 0 max 16384 2024-04-22T07:06:12.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.609+0000 7f524d5dd740 1 Processor -- start 2024-04-22T07:06:12.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.609+0000 7f524d5dd740 1 -- start start 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.610+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863175100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55586319b360 0x555863223510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631bfa90 0x555863223a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f524d5dd740 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5244166620 con 0x555863177a10 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f524d5dd740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5244143db0 con 0x5558631bfa90 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5243fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863175100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5243fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863175100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58290/0 (socket says 172.21.15.44:58290) 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5243fff640 1 -- 172.21.15.44:0/4090279538 learned_addr learned my addr 172.21.15.44:0/4090279538 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f524d5dd740 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f5244165f50 con 0x55586319b360 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f52437fe640 1 --2- 172.21.15.44:0/4090279538 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55586319b360 0x555863223510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5248f91640 1 --2- 172.21.15.44:0/4090279538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631bfa90 0x555863223a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5243fff640 1 -- 172.21.15.44:0/4090279538 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55586319b360 msgr2=0x555863223510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:12.611 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5243fff640 1 --2- 172.21.15.44:0/4090279538 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55586319b360 0x555863223510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5243fff640 1 -- 172.21.15.44:0/4090279538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631bfa90 msgr2=0x555863223a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:12.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5243fff640 1 --2- 172.21.15.44:0/4090279538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631bfa90 0x555863223a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5243fff640 1 -- 172.21.15.44:0/4090279538 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555863178b90 con 0x555863177a10 2024-04-22T07:06:12.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5248f91640 1 --2- 172.21.15.44:0/4090279538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631bfa90 0x555863223a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:06:12.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.611+0000 7f5243fff640 1 --2- 172.21.15.44:0/4090279538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863175100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f52340782f0 tx=0x7f5234050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6a2761d4d9e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.612+0000 7f52297fa640 1 -- 172.21.15.44:0/4090279538 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234080240 con 0x555863177a10 2024-04-22T07:06:12.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.612+0000 7f52297fa640 1 -- 172.21.15.44:0/4090279538 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523409b040 con 0x555863177a10 2024-04-22T07:06:12.612 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.612+0000 7f52297fa640 1 -- 172.21.15.44:0/4090279538 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523407be90 con 0x555863177a10 2024-04-22T07:06:12.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.613+0000 7f524d5dd740 1 -- 172.21.15.44:0/4090279538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 msgr2=0x555863175100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.613+0000 7f524d5dd740 1 --2- 172.21.15.44:0/4090279538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863175100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f52340782f0 tx=0x7f5234050000 comp rx=0 tx=0).stop 2024-04-22T07:06:12.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.613+0000 7f524d5dd740 1 -- 172.21.15.44:0/4090279538 shutdown_connections 2024-04-22T07:06:12.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.613+0000 7f524d5dd740 1 --2- 172.21.15.44:0/4090279538 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x55586319b360 0x555863223510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.613 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.613+0000 7f524d5dd740 1 --2- 172.21.15.44:0/4090279538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5558631bfa90 0x555863223a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.613+0000 7f524d5dd740 1 --2- 172.21.15.44:0/4090279538 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x555863175100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.613+0000 7f524d5dd740 1 -- 172.21.15.44:0/4090279538 >> 172.21.15.44:0/4090279538 conn(0x555863176470 msgr2=0x555863177df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.613+0000 7f524d5dd740 1 -- 172.21.15.44:0/4090279538 shutdown_connections 2024-04-22T07:06:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.613+0000 7f524d5dd740 1 -- 172.21.15.44:0/4090279538 wait complete. 2024-04-22T07:06:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.614+0000 7f524d5dd740 1 -- start start 2024-04-22T07:06:12.614 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.614+0000 7f524d5dd740 1 Processor -- start 2024-04-22T07:06:12.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631dab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55586319b360 0x5558632295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f524d5dd740 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 0x5558631912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f5243fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631dab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f5243fff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631dab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:58296/0 (socket says 172.21.15.44:58296) 2024-04-22T07:06:12.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f5243fff640 1 -- 172.21.15.44:0/2337657291 learned_addr learned my addr 172.21.15.44:0/2337657291 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:12.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f524d5dd740 1 -- 172.21.15.44:0/2337657291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f5244165880 con 0x555863177a10 2024-04-22T07:06:12.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f524d5dd740 1 -- 172.21.15.44:0/2337657291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5244144460 con 0x55586319b360 2024-04-22T07:06:12.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f52437fe640 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55586319b360 0x5558632295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.617 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f5248f91640 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 0x5558631912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.615+0000 7f524d5dd740 1 -- 172.21.15.44:0/2337657291 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x5558630e1300 con 0x5558631bfa90 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f524d5dd740 10 client.0 _ll_register_callbacks cb 0x7f523c001be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5243fff640 1 -- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 msgr2=0x5558631912c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5243fff640 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 0x5558631912c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5243fff640 1 -- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55586319b360 msgr2=0x5558632295d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5243fff640 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55586319b360 0x5558632295d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5243fff640 1 -- 172.21.15.44:0/2337657291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558631bc3c0 con 0x555863177a10 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5248f91640 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 0x5558631912c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5243fff640 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631dab20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f52340ae170 tx=0x7f5234009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5388dde330811d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5222ffd640 10 client.0 ms_handle_connect on v2:172.21.15.44:3300/0 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5222ffd640 1 -- 172.21.15.44:0/2337657291 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523407c590 con 0x555863177a10 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f524d5dd740 1 -- 172.21.15.44:0/2337657291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f523409b040 con 0x555863177a10 2024-04-22T07:06:12.618 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5222ffd640 1 -- 172.21.15.44:0/2337657291 <== mon.0 v2:172.21.15.44:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f52340845e0 con 0x555863177a10 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.616+0000 7f5222ffd640 1 -- 172.21.15.44:0/2337657291 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5234080240 con 0x555863177a10 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.617+0000 7f524d5dd740 10 client.5049 fetch_fsmap learned FSMap version 20 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.617+0000 7f524d5dd740 1 -- 172.21.15.44:0/2337657291 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x5558631bc3c0 con 0x555863177a10 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.617+0000 7f5222ffd640 1 -- 172.21.15.44:0/2337657291 <== mon.0 v2:172.21.15.44:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523409b040 con 0x555863177a10 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.618+0000 7f5222ffd640 1 -- 172.21.15.44:0/2337657291 <== mon.0 v2:172.21.15.44:3300/0 5 ==== fsmap(e 20) ==== 4171+0+0 (secure 0 0 0) 0x7f523409b1f0 con 0x555863177a10 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.618+0000 7f524d5dd740 10 client.5049 fetch_fsmap finished waiting for FSMap version 20 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.618+0000 7f524d5dd740 10 client.5049 resolve_mds: validated gid 4324 aka daemon mds.f 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.618+0000 7f524d5dd740 20 client.5049 populate_metadata read hostname 'smithi044' 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.618+0000 7f524d5dd740 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x5558631c1890 0x5558631bb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.618+0000 7f524d5dd740 4 client.5049 mds_command: new command op to 4324 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.6, "await": true}] 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.618+0000 7f524d5dd740 1 -- 172.21.15.44:0/2337657291 --> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.6, "await": true}) -- 0x555863174b70 con 0x5558631c1890 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.618+0000 7f52437fe640 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x5558631c1890 0x5558631bb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.618+0000 7f5222ffd640 10 client.5049 ms_handle_connect on v2:172.21.15.70:6836/2872131879 2024-04-22T07:06:12.619 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.618+0000 7f52437fe640 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x5558631c1890 0x5558631bb0d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c0eb98a05422be server_cookie=958fe601de4de789 in_seq=0 out_seq=0 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f5222ffd640 1 -- 172.21.15.44:0/2337657291 <== mds.0 v2:172.21.15.70:6836/2872131879 1 ==== command_reply(tid 0: -9 ) ==== 8+0+583 (crc 0 0 0) 0x555863174b70 con 0x5558631c1890 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f5222ffd640 10 client.5049 handle_command_reply: tid=0 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f524d5dd740 1 client.5049 shutdown 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f524d5dd740 1 -- 172.21.15.44:0/2337657291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 msgr2=0x5558631dab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f524d5dd740 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631dab20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f52340ae170 tx=0x7f5234009240 comp rx=0 tx=0).stop 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f524d5dd740 1 -- 172.21.15.44:0/2337657291 shutdown_connections 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f524d5dd740 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x5558631bfa90 0x5558631912c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f524d5dd740 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:6836/2872131879,v1:172.21.15.70:6837/2872131879] conn(0x5558631c1890 0x5558631bb0d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f524d5dd740 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55586319b360 0x5558632295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f524d5dd740 1 --2- 172.21.15.44:0/2337657291 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x555863177a10 0x5558631dab20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:12.621 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f524d5dd740 1 -- 172.21.15.44:0/2337657291 >> 172.21.15.44:0/2337657291 conn(0x555863176470 msgr2=0x555863175100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:12.622 INFO:teuthology.orchestra.run.smithi044.stdout:{ 2024-04-22T07:06:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "epoch": 20, 2024-04-22T07:06:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "leader": 4324, 2024-04-22T07:06:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "set_version": 6, 2024-04-22T07:06:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "sets": { 2024-04-22T07:06:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "11897366": { 2024-04-22T07:06:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "version": 6, 2024-04-22T07:06:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "age_ref": 0.0, 2024-04-22T07:06:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T07:06:12.622 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FAILED", 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: }, 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "timeout": 60.0, 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "expiration": 79.6, 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "members": { 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "file:/": { 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "excluded": false, 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "state": { 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "name": "FAILED", 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: "age": 0.0 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:06:12.623 INFO:teuthology.orchestra.run.smithi044.stdout: } 2024-04-22T07:06:12.624 INFO:teuthology.orchestra.run.smithi044.stdout:} 2024-04-22T07:06:12.624 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-22T07:06:12.625 INFO:teuthology.orchestra.run.smithi044.stderr:Error EBADF: 2024-04-22T07:06:12.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f5222ffd640 0 client.5049 ms_handle_reset on v2:172.21.15.70:6836/2872131879 2024-04-22T07:06:12.625 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:12.619+0000 7f524d5dd740 1 -- 172.21.15.44:0/2337657291 shutdown_connections 2024-04-22T07:06:12.625 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't quiesce root with rc: -9 (EBADF), stdout: { "epoch": 20, "leader": 4324, "set_version": 6, "sets": { "11897366": { "version": 6, "age_ref": 0.0, "state": { "name": "FAILED", "age": 0.0 }, "timeout": 60.0, "expiration": 79.6, "members": { "file:/": { "excluded": false, "state": { "name": "FAILED", "age": 0.0 } } } } } } 2024-04-22T07:06:12.625 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 240, in _run self.do_quiesce(d) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 197, in do_quiesce raise RuntimeError(f"Error quiescing root: {rcinfo}") RuntimeError: Error quiescing root: -9 (EBADF) 2024-04-22T07:06:16.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~39s 2024-04-22T07:06:16.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~39s 2024-04-22T07:06:16.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~39s 2024-04-22T07:06:16.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~39s 2024-04-22T07:06:16.812 INFO:tasks.daemonwatchdog.daemon_watchdog:quiescer.fs.[cephfs] failed 2024-04-22T07:06:16.812 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-04-22T07:06:16.812 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:06:16.813 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T07:06:16.835 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:06:17.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.032+0000 7fc60d6b3640 1 Processor -- start 2024-04-22T07:06:17.031 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.032+0000 7fc60d6b3640 1 -- start start 2024-04-22T07:06:17.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc60d6b3640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc608154f70 0x7fc6080d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc60d6b3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc608155350 0x7fc6080d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc60d6b3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc60815b180 con 0x7fc608072d10 2024-04-22T07:06:17.033 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc60d6b3640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fc60815b460 con 0x7fc608154f70 2024-04-22T07:06:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc60d6b3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fc60815b740 con 0x7fc608155350 2024-04-22T07:06:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc606ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc608154f70 0x7fc6080d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc606ffd640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fc608154f70 0x7fc6080d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42786/0 (socket says 172.21.15.44:42786) 2024-04-22T07:06:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc606ffd640 1 -- 172.21.15.44:0/3152680642 learned_addr learned my addr 172.21.15.44:0/3152680642 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc6067fc640 1 --2- 172.21.15.44:0/3152680642 >> v2:172.21.15.70:3300/0 conn(0x7fc608155350 0x7fc6080d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc606ffd640 1 -- 172.21.15.44:0/3152680642 >> v2:172.21.15.70:3300/0 conn(0x7fc608155350 msgr2=0x7fc6080d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc606ffd640 1 --2- 172.21.15.44:0/3152680642 >> v2:172.21.15.70:3300/0 conn(0x7fc608155350 0x7fc6080d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc606ffd640 1 -- 172.21.15.44:0/3152680642 >> v1:172.21.15.70:6789/0 conn(0x7fc608072d10 legacy=0x7fc6080b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.034 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.034+0000 7fc606ffd640 1 -- 172.21.15.44:0/3152680642 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60815ba20 con 0x7fc608154f70 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.035+0000 7fc606ffd640 1 --2- 172.21.15.44:0/3152680642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc608154f70 0x7fc6080d0900 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc6080b3ca0 tx=0x7fc5f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8420275aa6e76a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.036+0000 7fc605ffb640 1 -- 172.21.15.44:0/3152680642 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8034070 con 0x7fc608154f70 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.036+0000 7fc605ffb640 1 -- 172.21.15.44:0/3152680642 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f8030860 con 0x7fc608154f70 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.036+0000 7fc605ffb640 1 -- 172.21.15.44:0/3152680642 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8030c30 con 0x7fc608154f70 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.037+0000 7fc60d6b3640 1 -- 172.21.15.44:0/3152680642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc608154f70 msgr2=0x7fc6080d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.037+0000 7fc60d6b3640 1 --2- 172.21.15.44:0/3152680642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc608154f70 0x7fc6080d0900 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc6080b3ca0 tx=0x7fc5f802b9a0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.037+0000 7fc60d6b3640 1 -- 172.21.15.44:0/3152680642 shutdown_connections 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.037+0000 7fc60d6b3640 1 --2- 172.21.15.44:0/3152680642 >> v2:172.21.15.70:3300/0 conn(0x7fc608155350 0x7fc6080d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.037+0000 7fc60d6b3640 1 --2- 172.21.15.44:0/3152680642 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc608154f70 0x7fc6080d0900 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.037+0000 7fc60d6b3640 1 -- 172.21.15.44:0/3152680642 >> 172.21.15.44:0/3152680642 conn(0x7fc6080735c0 msgr2=0x7fc6080739d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:17.036 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.037+0000 7fc60d6b3640 1 -- 172.21.15.44:0/3152680642 shutdown_connections 2024-04-22T07:06:17.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.037+0000 7fc60d6b3640 1 -- 172.21.15.44:0/3152680642 wait complete. 2024-04-22T07:06:17.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.039+0000 7fc60d6b3640 1 Processor -- start 2024-04-22T07:06:17.037 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.039+0000 7fc60d6b3640 1 -- start start 2024-04-22T07:06:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc60d6b3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608072d10 0x7fc6081e58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc60d6b3640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6081e6410 0x7fc6081e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc60d6b3640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60814fad0 0x7fc60814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc60d6b3640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7fc60815add0 con 0x7fc60814fad0 2024-04-22T07:06:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc60d6b3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc6080caf30 con 0x7fc608072d10 2024-04-22T07:06:17.040 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc60d6b3640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fc6080cc750 con 0x7fc6081e6410 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc6077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60814fad0 0x7fc60814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc6077fe640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60814fad0 0x7fc60814ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42800/0 (socket says 172.21.15.44:42800) 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc6077fe640 1 -- 172.21.15.44:0/2548567140 learned_addr learned my addr 172.21.15.44:0/2548567140 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc6067fc640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6081e6410 0x7fc6081e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.040+0000 7fc606ffd640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608072d10 0x7fc6081e58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc6077fe640 1 -- 172.21.15.44:0/2548567140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6081e6410 msgr2=0x7fc6081e5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc6077fe640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6081e6410 0x7fc6081e5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc6077fe640 1 -- 172.21.15.44:0/2548567140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608072d10 msgr2=0x7fc6081e58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc6077fe640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608072d10 0x7fc6081e58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc6077fe640 1 -- 172.21.15.44:0/2548567140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc608150490 con 0x7fc60814fad0 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc6077fe640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60814fad0 0x7fc60814ff50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc60000c760 tx=0x7fc60000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc5ef7fe640 1 -- 172.21.15.44:0/2548567140 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600016020 con 0x7fc60814fad0 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc5ef7fe640 1 -- 172.21.15.44:0/2548567140 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc600006d50 con 0x7fc60814fad0 2024-04-22T07:06:17.041 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc5ef7fe640 1 -- 172.21.15.44:0/2548567140 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600003bb0 con 0x7fc60814fad0 2024-04-22T07:06:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc60d6b3640 1 -- 172.21.15.44:0/2548567140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6081e69c0 con 0x7fc60814fad0 2024-04-22T07:06:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.041+0000 7fc60d6b3640 1 -- 172.21.15.44:0/2548567140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6080ccc80 con 0x7fc60814fad0 2024-04-22T07:06:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.043+0000 7fc60d6b3640 1 -- 172.21.15.44:0/2548567140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc608069ea0 con 0x7fc60814fad0 2024-04-22T07:06:17.042 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.043+0000 7fc5ef7fe640 1 -- 172.21.15.44:0/2548567140 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc60000a7f0 con 0x7fc60814fad0 2024-04-22T07:06:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.043+0000 7fc5ef7fe640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc5f406c920 0x7fc5f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.043+0000 7fc5ef7fe640 1 -- 172.21.15.44:0/2548567140 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7fc60008b690 con 0x7fc60814fad0 2024-04-22T07:06:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.044+0000 7fc606ffd640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc5f406c920 0x7fc5f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.043 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.044+0000 7fc606ffd640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc5f406c920 0x7fc5f406ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8036000 tx=0x7fc5f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:17.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.052+0000 7fc5ef7fe640 1 -- 172.21.15.44:0/2548567140 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc600020020 con 0x7fc60814fad0 2024-04-22T07:06:17.237 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.238+0000 7fc60d6b3640 1 -- 172.21.15.44:0/2548567140 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc60814f410 con 0x7fc60814fad0 2024-04-22T07:06:17.239 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.240+0000 7fc5ef7fe640 1 -- 172.21.15.44:0/2548567140 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+16936 (secure 0 0 0) 0x7fc608150490 con 0x7fc60814fad0 2024-04-22T07:06:17.239 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:06:17.239 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:05:49.227595+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6834/1969018412":"2024-04-23T07:05:49.227539+0000","172.21.15.70:6832/1969018412":"2024-04-23T07:05:49.227539+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:06:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 -- 172.21.15.44:0/2548567140 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc5f406c920 msgr2=0x7fc5f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc5f406c920 0x7fc5f406ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8036000 tx=0x7fc5f8008030 comp rx=0 tx=0).stop 2024-04-22T07:06:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 -- 172.21.15.44:0/2548567140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60814fad0 msgr2=0x7fc60814ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60814fad0 0x7fc60814ff50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc60000c760 tx=0x7fc60000cc20 comp rx=0 tx=0).stop 2024-04-22T07:06:17.241 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 -- 172.21.15.44:0/2548567140 shutdown_connections 2024-04-22T07:06:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7fc5f406c920 0x7fc5f406ede0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fc6081e6410 0x7fc6081e5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7fc60814fad0 0x7fc60814ff50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 --2- 172.21.15.44:0/2548567140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608072d10 0x7fc6081e58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 -- 172.21.15.44:0/2548567140 >> 172.21.15.44:0/2548567140 conn(0x7fc6080735c0 msgr2=0x7fc6080ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 -- 172.21.15.44:0/2548567140 shutdown_connections 2024-04-22T07:06:17.242 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.242+0000 7fc60d6b3640 1 -- 172.21.15.44:0/2548567140 wait complete. 2024-04-22T07:06:17.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:04:01.723984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:11.652423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:12.098854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:06:17.256 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:06:17.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.362+0000 7f0038d0e640 1 Processor -- start 2024-04-22T07:06:17.361 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.362+0000 7f0038d0e640 1 -- start start 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.363+0000 7f0038d0e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f003406d5d0 0x7f00340c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.363+0000 7f0038d0e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f003406d9b0 0x7f0034151530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.363+0000 7f0038d0e640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00340c5560 con 0x7f0034071770 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.363+0000 7f0038d0e640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f00340c5840 con 0x7f003406d9b0 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.363+0000 7f0038d0e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f00340c5b20 con 0x7f003406d5d0 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.363+0000 7f0032d9d640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f0034071770 0x7f0034157940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:32794/0 (socket says 172.21.15.44:32794) 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.363+0000 7f0032d9d640 1 -- 172.21.15.44:0/671243141 learned_addr learned my addr 172.21.15.44:0/671243141 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.364+0000 7f003259c640 1 --2- 172.21.15.44:0/671243141 >> v2:172.21.15.70:3300/0 conn(0x7f003406d5d0 0x7f00340c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.363+0000 7f0031d9b640 1 --2- 172.21.15.44:0/671243141 >> v2:172.21.15.44:3300/0 conn(0x7f003406d9b0 0x7f0034151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.364+0000 7f003159a640 1 -- 172.21.15.44:0/671243141 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2852811916 0 0) 0x7f00340c5560 con 0x7f0034071770 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.364+0000 7f003159a640 1 -- 172.21.15.44:0/671243141 --> v1:172.21.15.44:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f001c003520 con 0x7f0034071770 2024-04-22T07:06:17.363 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.364+0000 7f0031d9b640 1 -- 172.21.15.44:0/671243141 >> v2:172.21.15.70:3300/0 conn(0x7f003406d5d0 msgr2=0x7f00340c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.364+0000 7f0031d9b640 1 --2- 172.21.15.44:0/671243141 >> v2:172.21.15.70:3300/0 conn(0x7f003406d5d0 0x7f00340c3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.364+0000 7f0031d9b640 1 -- 172.21.15.44:0/671243141 >> v1:172.21.15.44:6789/0 conn(0x7f0034071770 legacy=0x7f0034157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.364 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.364+0000 7f0031d9b640 1 -- 172.21.15.44:0/671243141 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0034073130 con 0x7f003406d9b0 2024-04-22T07:06:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.365+0000 7f0031d9b640 1 --2- 172.21.15.44:0/671243141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d9b0 0x7f0034151530 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f00280097a0 tx=0x7f002802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102abdad0b6aad3e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.365+0000 7f003159a640 1 -- 172.21.15.44:0/671243141 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028034070 con 0x7f003406d9b0 2024-04-22T07:06:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.365+0000 7f003159a640 1 -- 172.21.15.44:0/671243141 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0028030610 con 0x7f003406d9b0 2024-04-22T07:06:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.365+0000 7f003159a640 1 -- 172.21.15.44:0/671243141 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00280309c0 con 0x7f003406d9b0 2024-04-22T07:06:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.366+0000 7f0038d0e640 1 -- 172.21.15.44:0/671243141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d9b0 msgr2=0x7f0034151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.366+0000 7f0038d0e640 1 --2- 172.21.15.44:0/671243141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d9b0 0x7f0034151530 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f00280097a0 tx=0x7f002802b960 comp rx=0 tx=0).stop 2024-04-22T07:06:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.366+0000 7f0038d0e640 1 -- 172.21.15.44:0/671243141 shutdown_connections 2024-04-22T07:06:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.366+0000 7f0038d0e640 1 --2- 172.21.15.44:0/671243141 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d9b0 0x7f0034151530 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.365 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.366+0000 7f0038d0e640 1 --2- 172.21.15.44:0/671243141 >> v2:172.21.15.70:3300/0 conn(0x7f003406d5d0 0x7f00340c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.366+0000 7f0038d0e640 1 -- 172.21.15.44:0/671243141 >> 172.21.15.44:0/671243141 conn(0x7f00340c9020 msgr2=0x7f00340c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:17.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.366+0000 7f0038d0e640 1 -- 172.21.15.44:0/671243141 shutdown_connections 2024-04-22T07:06:17.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.366+0000 7f0038d0e640 1 -- 172.21.15.44:0/671243141 wait complete. 2024-04-22T07:06:17.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.367+0000 7f0038d0e640 1 Processor -- start 2024-04-22T07:06:17.366 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.367+0000 7f0038d0e640 1 -- start start 2024-04-22T07:06:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.368+0000 7f0038d0e640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d5d0 0x7f003414ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.368+0000 7f0038d0e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003406d9b0 0x7f00340cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.368+0000 7f0038d0e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0034071770 0x7f0034162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.368+0000 7f0038d0e640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f00340c3b40 con 0x7f003406d5d0 2024-04-22T07:06:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.368+0000 7f0038d0e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f00340736f0 con 0x7f003406d9b0 2024-04-22T07:06:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.368+0000 7f0038d0e640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f00340c5200 con 0x7f0034071770 2024-04-22T07:06:17.368 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.368+0000 7f003259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d5d0 0x7f003414ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.368+0000 7f003259c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d5d0 0x7f003414ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42820/0 (socket says 172.21.15.44:42820) 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.368+0000 7f003259c640 1 -- 172.21.15.44:0/2155304633 learned_addr learned my addr 172.21.15.44:0/2155304633 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f0031d9b640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003406d9b0 0x7f00340cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f0032d9d640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0034071770 0x7f0034162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f003259c640 1 -- 172.21.15.44:0/2155304633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0034071770 msgr2=0x7f0034162ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f003259c640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0034071770 0x7f0034162ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f003259c640 1 -- 172.21.15.44:0/2155304633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003406d9b0 msgr2=0x7f00340cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f003259c640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003406d9b0 0x7f00340cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f003259c640 1 -- 172.21.15.44:0/2155304633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003414bf80 con 0x7f003406d5d0 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f003259c640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d5d0 0x7f003414ba40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f002400d620 tx=0x7f002400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f0022ffd640 1 -- 172.21.15.44:0/2155304633 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00240125f0 con 0x7f003406d5d0 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f0022ffd640 1 -- 172.21.15.44:0/2155304633 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0024019070 con 0x7f003406d5d0 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f0038d0e640 1 -- 172.21.15.44:0/2155304633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00340ca340 con 0x7f003406d5d0 2024-04-22T07:06:17.369 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f0022ffd640 1 -- 172.21.15.44:0/2155304633 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00240129d0 con 0x7f003406d5d0 2024-04-22T07:06:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.369+0000 7f0038d0e640 1 -- 172.21.15.44:0/2155304633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00341623f0 con 0x7f003406d5d0 2024-04-22T07:06:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.371+0000 7f0038d0e640 1 -- 172.21.15.44:0/2155304633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0034068b00 con 0x7f003406d5d0 2024-04-22T07:06:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.371+0000 7f0022ffd640 1 -- 172.21.15.44:0/2155304633 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0024012b70 con 0x7f003406d5d0 2024-04-22T07:06:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.371+0000 7f0022ffd640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.370 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.372+0000 7f0022ffd640 1 -- 172.21.15.44:0/2155304633 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f002408be00 con 0x7f003406d5d0 2024-04-22T07:06:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.372+0000 7f0031d9b640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.371 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.373+0000 7f0031d9b640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f001406c8f0 0x7f001406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0028036000 tx=0x7f0028008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:17.375 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.376+0000 7f0022ffd640 1 -- 172.21.15.44:0/2155304633 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0024059ba0 con 0x7f003406d5d0 2024-04-22T07:06:17.562 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.564+0000 7f0038d0e640 1 -- 172.21.15.44:0/2155304633 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f003406f9e0 con 0x7f003406d5d0 2024-04-22T07:06:17.563 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.564+0000 7f0022ffd640 1 -- 172.21.15.44:0/2155304633 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f002405dea0 con 0x7f003406d5d0 2024-04-22T07:06:17.563 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T07:06:17.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.566+0000 7f0038d0e640 1 -- 172.21.15.44:0/2155304633 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f001406c8f0 msgr2=0x7f001406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.566+0000 7f0038d0e640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f001406c8f0 0x7f001406edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0028036000 tx=0x7f0028008030 comp rx=0 tx=0).stop 2024-04-22T07:06:17.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.566+0000 7f0038d0e640 1 -- 172.21.15.44:0/2155304633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d5d0 msgr2=0x7f003414ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.566+0000 7f0038d0e640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d5d0 0x7f003414ba40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f002400d620 tx=0x7f002400dae0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.567+0000 7f0038d0e640 1 -- 172.21.15.44:0/2155304633 shutdown_connections 2024-04-22T07:06:17.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.567+0000 7f0038d0e640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f001406c8f0 0x7f001406edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.567+0000 7f0038d0e640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f0034071770 0x7f0034162ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.567+0000 7f0038d0e640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003406d9b0 0x7f00340cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.567+0000 7f0038d0e640 1 --2- 172.21.15.44:0/2155304633 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f003406d5d0 0x7f003414ba40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.567+0000 7f0038d0e640 1 -- 172.21.15.44:0/2155304633 >> 172.21.15.44:0/2155304633 conn(0x7f00340c9020 msgr2=0x7f0034151300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:17.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.567+0000 7f0038d0e640 1 -- 172.21.15.44:0/2155304633 shutdown_connections 2024-04-22T07:06:17.566 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.567+0000 7f0038d0e640 1 -- 172.21.15.44:0/2155304633 wait complete. 2024-04-22T07:06:17.580 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:06:17.685 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.687+0000 7f739b9f4640 1 Processor -- start 2024-04-22T07:06:17.686 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.687+0000 7f739b9f4640 1 -- start start 2024-04-22T07:06:17.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f739b9f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73941552c0 0x7f73940b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f739b9f4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f73940caec0 0x7f73940d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f739b9f4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73941556a0 con 0x7f73940cc920 2024-04-22T07:06:17.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f739b9f4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f739415b160 con 0x7f73940caec0 2024-04-22T07:06:17.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f739b9f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f739415b440 con 0x7f73941552c0 2024-04-22T07:06:17.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f7398f8f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f73940caec0 0x7f73940d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.687 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f7398f8f640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f73940caec0 0x7f73940d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42832/0 (socket says 172.21.15.44:42832) 2024-04-22T07:06:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f7398f8f640 1 -- 172.21.15.44:0/2570986118 learned_addr learned my addr 172.21.15.44:0/2570986118 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f7399790640 1 --2- 172.21.15.44:0/2570986118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73941552c0 0x7f73940b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f7398f8f640 1 -- 172.21.15.44:0/2570986118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73941552c0 msgr2=0x7f73940b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f7398f8f640 1 --2- 172.21.15.44:0/2570986118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73941552c0 0x7f73940b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f738bfff640 1 -- 172.21.15.44:0/2570986118 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1396658162 0 0) 0x7f73941556a0 con 0x7f73940cc920 2024-04-22T07:06:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f7398f8f640 1 -- 172.21.15.44:0/2570986118 >> v1:172.21.15.70:6789/0 conn(0x7f73940cc920 legacy=0x7f73940cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.688 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.688+0000 7f7398f8f640 1 -- 172.21.15.44:0/2570986118 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739415b720 con 0x7f73940caec0 2024-04-22T07:06:17.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.690+0000 7f7398f8f640 1 --2- 172.21.15.44:0/2570986118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73940caec0 0x7f73940d0a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7384009870 tx=0x7f738402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c298e155fb837cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:17.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.690+0000 7f738bfff640 1 -- 172.21.15.44:0/2570986118 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384034070 con 0x7f73940caec0 2024-04-22T07:06:17.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.690+0000 7f738bfff640 1 -- 172.21.15.44:0/2570986118 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7384030840 con 0x7f73940caec0 2024-04-22T07:06:17.689 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.690+0000 7f738bfff640 1 -- 172.21.15.44:0/2570986118 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384030c10 con 0x7f73940caec0 2024-04-22T07:06:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.691+0000 7f739b9f4640 1 -- 172.21.15.44:0/2570986118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73940caec0 msgr2=0x7f73940d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.691+0000 7f739b9f4640 1 --2- 172.21.15.44:0/2570986118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73940caec0 0x7f73940d0a20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7384009870 tx=0x7f738402b980 comp rx=0 tx=0).stop 2024-04-22T07:06:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.691+0000 7f739b9f4640 1 -- 172.21.15.44:0/2570986118 shutdown_connections 2024-04-22T07:06:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.691+0000 7f739b9f4640 1 --2- 172.21.15.44:0/2570986118 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73940caec0 0x7f73940d0a20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.691+0000 7f739b9f4640 1 --2- 172.21.15.44:0/2570986118 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73941552c0 0x7f73940b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.691+0000 7f739b9f4640 1 -- 172.21.15.44:0/2570986118 >> 172.21.15.44:0/2570986118 conn(0x7f73940676b0 msgr2=0x7f7394067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.691+0000 7f739b9f4640 1 -- 172.21.15.44:0/2570986118 shutdown_connections 2024-04-22T07:06:17.690 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.691+0000 7f739b9f4640 1 -- 172.21.15.44:0/2570986118 wait complete. 2024-04-22T07:06:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.693+0000 7f739b9f4640 1 Processor -- start 2024-04-22T07:06:17.691 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.693+0000 7f739b9f4640 1 -- start start 2024-04-22T07:06:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f739b9f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73940caec0 0x7f73941f1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f739b9f4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73940cc920 0x7f7394150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f739b9f4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73941552c0 0x7f739415c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f739b9f4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f73940c6a00 con 0x7f73941552c0 2024-04-22T07:06:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f739b9f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f739415bce0 con 0x7f73940caec0 2024-04-22T07:06:17.694 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f739b9f4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f73940cda20 con 0x7f73940cc920 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f7399f91640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73941552c0 0x7f739415c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f7399f91640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73941552c0 0x7f739415c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42848/0 (socket says 172.21.15.44:42848) 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f7399f91640 1 -- 172.21.15.44:0/2824572121 learned_addr learned my addr 172.21.15.44:0/2824572121 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f7399790640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73940caec0 0x7f73941f1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.694+0000 7f7398f8f640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73940cc920 0x7f7394150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f7399f91640 1 -- 172.21.15.44:0/2824572121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73940cc920 msgr2=0x7f7394150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f7399f91640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73940cc920 0x7f7394150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f7399f91640 1 -- 172.21.15.44:0/2824572121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73940caec0 msgr2=0x7f73941f1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f7399f91640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73940caec0 0x7f73941f1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f7399f91640 1 -- 172.21.15.44:0/2824572121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73941515c0 con 0x7f73941552c0 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f7398f8f640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73940cc920 0x7f7394150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f7399f91640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73941552c0 0x7f739415c580 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f738c00c7e0 tx=0x7f738c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f7389ffb640 1 -- 172.21.15.44:0/2824572121 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c017070 con 0x7f73941552c0 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f7389ffb640 1 -- 172.21.15.44:0/2824572121 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738c003dc0 con 0x7f73941552c0 2024-04-22T07:06:17.695 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f739b9f4640 1 -- 172.21.15.44:0/2824572121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73941518a0 con 0x7f73941552c0 2024-04-22T07:06:17.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f7389ffb640 1 -- 172.21.15.44:0/2824572121 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738c0052b0 con 0x7f73941552c0 2024-04-22T07:06:17.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.695+0000 7f739b9f4640 1 -- 172.21.15.44:0/2824572121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739414fca0 con 0x7f73941552c0 2024-04-22T07:06:17.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.697+0000 7f739b9f4640 1 -- 172.21.15.44:0/2824572121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7394068b00 con 0x7f73941552c0 2024-04-22T07:06:17.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.697+0000 7f7389ffb640 1 -- 172.21.15.44:0/2824572121 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f738c01a070 con 0x7f73941552c0 2024-04-22T07:06:17.696 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.698+0000 7f7389ffb640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f737806c8f0 0x7f737806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:17.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.698+0000 7f7389ffb640 1 -- 172.21.15.44:0/2824572121 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f738c08b2d0 con 0x7f73941552c0 2024-04-22T07:06:17.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.698+0000 7f7399790640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f737806c8f0 0x7f737806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:17.697 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.698+0000 7f7399790640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f737806c8f0 0x7f737806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7390009aa0 tx=0x7f7390009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:17.704 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.706+0000 7f7389ffb640 1 -- 172.21.15.44:0/2824572121 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f738c058f50 con 0x7f73941552c0 2024-04-22T07:06:17.891 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.892+0000 7f739b9f4640 1 -- 172.21.15.44:0/2824572121 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7394069330 con 0x7f73941552c0 2024-04-22T07:06:17.892 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.892+0000 7f7389ffb640 1 -- 172.21.15.44:0/2824572121 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f738c05d250 con 0x7f73941552c0 2024-04-22T07:06:17.892 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T07:06:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.894+0000 7f739b9f4640 1 -- 172.21.15.44:0/2824572121 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f737806c8f0 msgr2=0x7f737806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.894+0000 7f739b9f4640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f737806c8f0 0x7f737806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7390009aa0 tx=0x7f7390009190 comp rx=0 tx=0).stop 2024-04-22T07:06:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.895+0000 7f739b9f4640 1 -- 172.21.15.44:0/2824572121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73941552c0 msgr2=0x7f739415c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.895+0000 7f739b9f4640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73941552c0 0x7f739415c580 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f738c00c7e0 tx=0x7f738c00cca0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.895+0000 7f739b9f4640 1 -- 172.21.15.44:0/2824572121 shutdown_connections 2024-04-22T07:06:17.894 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.895+0000 7f739b9f4640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f737806c8f0 0x7f737806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.895+0000 7f739b9f4640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f73940cc920 0x7f7394150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.895+0000 7f739b9f4640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f73941552c0 0x7f739415c580 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.895+0000 7f739b9f4640 1 --2- 172.21.15.44:0/2824572121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73940caec0 0x7f73941f1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.895+0000 7f739b9f4640 1 -- 172.21.15.44:0/2824572121 >> 172.21.15.44:0/2824572121 conn(0x7f73940676b0 msgr2=0x7f73940c5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.895+0000 7f739b9f4640 1 -- 172.21.15.44:0/2824572121 shutdown_connections 2024-04-22T07:06:17.895 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:17.895+0000 7f739b9f4640 1 -- 172.21.15.44:0/2824572121 wait complete. 2024-04-22T07:06:17.905 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:06:18.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.010+0000 7f544c992640 1 Processor -- start 2024-04-22T07:06:18.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.010+0000 7f544c992640 1 -- start start 2024-04-22T07:06:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544c992640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f54480cb300 0x7f5448071300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544c992640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5448067dc0 0x7f5448071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544c992640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 0x7f544815e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544c992640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f544815b0b0 con 0x7f54480cb300 2024-04-22T07:06:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544c992640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f544815b390 con 0x7f5448067dc0 2024-04-22T07:06:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544c992640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f544815b670 con 0x7f54480cb6e0 2024-04-22T07:06:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544659c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f54480cb300 0x7f5448071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544659c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f54480cb300 0x7f5448071300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42854/0 (socket says 172.21.15.44:42854) 2024-04-22T07:06:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544659c640 1 -- 172.21.15.44:0/2847340996 learned_addr learned my addr 172.21.15.44:0/2847340996 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:18.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f5446d9d640 1 --2- 172.21.15.44:0/2847340996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 0x7f544815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f5445d9b640 1 --2- 172.21.15.44:0/2847340996 >> v2:172.21.15.70:3300/0 conn(0x7f5448067dc0 0x7f5448071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544659c640 1 -- 172.21.15.44:0/2847340996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 msgr2=0x7f544815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:18.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544659c640 1 --2- 172.21.15.44:0/2847340996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 0x7f544815e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544659c640 1 -- 172.21.15.44:0/2847340996 >> v2:172.21.15.70:3300/0 conn(0x7f5448067dc0 msgr2=0x7f5448071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:18.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544659c640 1 --2- 172.21.15.44:0/2847340996 >> v2:172.21.15.70:3300/0 conn(0x7f5448067dc0 0x7f5448071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f544659c640 1 -- 172.21.15.44:0/2847340996 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544815b950 con 0x7f54480cb300 2024-04-22T07:06:18.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.012+0000 7f5446d9d640 1 --2- 172.21.15.44:0/2847340996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 0x7f544815e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:06:18.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.013+0000 7f544659c640 1 --2- 172.21.15.44:0/2847340996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54480cb300 0x7f5448071300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f544806db50 tx=0x7f543802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a174baeee768c73e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.013+0000 7f544559a640 1 -- 172.21.15.44:0/2847340996 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438034070 con 0x7f54480cb300 2024-04-22T07:06:18.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.013+0000 7f544559a640 1 -- 172.21.15.44:0/2847340996 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5438030860 con 0x7f54480cb300 2024-04-22T07:06:18.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.013+0000 7f544559a640 1 -- 172.21.15.44:0/2847340996 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438034070 con 0x7f54480cb300 2024-04-22T07:06:18.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.014+0000 7f544c992640 1 -- 172.21.15.44:0/2847340996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54480cb300 msgr2=0x7f5448071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.014+0000 7f544c992640 1 --2- 172.21.15.44:0/2847340996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54480cb300 0x7f5448071300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f544806db50 tx=0x7f543802b9a0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.014+0000 7f544c992640 1 -- 172.21.15.44:0/2847340996 shutdown_connections 2024-04-22T07:06:18.013 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.014+0000 7f544c992640 1 --2- 172.21.15.44:0/2847340996 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 0x7f544815e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.014+0000 7f544c992640 1 --2- 172.21.15.44:0/2847340996 >> v2:172.21.15.70:3300/0 conn(0x7f5448067dc0 0x7f5448071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.014+0000 7f544c992640 1 --2- 172.21.15.44:0/2847340996 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f54480cb300 0x7f5448071300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.014+0000 7f544c992640 1 -- 172.21.15.44:0/2847340996 >> 172.21.15.44:0/2847340996 conn(0x7f544806fa50 msgr2=0x7f54480676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:18.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.014+0000 7f544c992640 1 -- 172.21.15.44:0/2847340996 shutdown_connections 2024-04-22T07:06:18.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.014+0000 7f544c992640 1 -- 172.21.15.44:0/2847340996 wait complete. 2024-04-22T07:06:18.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.016+0000 7f544c992640 1 Processor -- start 2024-04-22T07:06:18.014 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.016+0000 7f544c992640 1 -- start start 2024-04-22T07:06:18.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544c992640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5448067dc0 0x7f5448153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544c992640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54480cb300 0x7f5448156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544c992640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 0x7f54480c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544c992640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f544815eb50 con 0x7f5448067dc0 2024-04-22T07:06:18.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544c992640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f544815ad80 con 0x7f54480cb300 2024-04-22T07:06:18.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544c992640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f544815ecd0 con 0x7f54480cb6e0 2024-04-22T07:06:18.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5448067dc0 0x7f5448153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.016 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544659c640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5448067dc0 0x7f5448153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42856/0 (socket says 172.21.15.44:42856) 2024-04-22T07:06:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544659c640 1 -- 172.21.15.44:0/3563288542 learned_addr learned my addr 172.21.15.44:0/3563288542 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f5446d9d640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 0x7f54480c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f5445d9b640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54480cb300 0x7f5448156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544659c640 1 -- 172.21.15.44:0/3563288542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 msgr2=0x7f54480c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544659c640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 0x7f54480c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544659c640 1 -- 172.21.15.44:0/3563288542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54480cb300 msgr2=0x7f5448156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544659c640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54480cb300 0x7f5448156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f544659c640 1 -- 172.21.15.44:0/3563288542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54481576c0 con 0x7f5448067dc0 2024-04-22T07:06:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.017+0000 7f5446d9d640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 0x7f54480c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:06:18.017 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.018+0000 7f544659c640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5448067dc0 0x7f5448153ae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f543802b9e0 tx=0x7f5438004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.018+0000 7f5436ffd640 1 -- 172.21.15.44:0/3563288542 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438049070 con 0x7f5448067dc0 2024-04-22T07:06:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.018+0000 7f5436ffd640 1 -- 172.21.15.44:0/3563288542 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5438039d90 con 0x7f5448067dc0 2024-04-22T07:06:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.018+0000 7f544c992640 1 -- 172.21.15.44:0/3563288542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54481579a0 con 0x7f5448067dc0 2024-04-22T07:06:18.018 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.019+0000 7f5436ffd640 1 -- 172.21.15.44:0/3563288542 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438034070 con 0x7f5448067dc0 2024-04-22T07:06:18.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.019+0000 7f544c992640 1 -- 172.21.15.44:0/3563288542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54480c3400 con 0x7f5448067dc0 2024-04-22T07:06:18.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.020+0000 7f544c992640 1 -- 172.21.15.44:0/3563288542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5448068b00 con 0x7f5448067dc0 2024-04-22T07:06:18.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.020+0000 7f5436ffd640 1 -- 172.21.15.44:0/3563288542 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f543804c070 con 0x7f5448067dc0 2024-04-22T07:06:18.019 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.020+0000 7f5436ffd640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f542806c8f0 0x7f542806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.021+0000 7f5445d9b640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f542806c8f0 0x7f542806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.021+0000 7f5445d9b640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f542806c8f0 0x7f542806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5448151ad0 tx=0x7f543c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.020 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.021+0000 7f5436ffd640 1 -- 172.21.15.44:0/3563288542 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f54380ac610 con 0x7f5448067dc0 2024-04-22T07:06:18.023 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.025+0000 7f5436ffd640 1 -- 172.21.15.44:0/3563288542 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f543807c400 con 0x7f5448067dc0 2024-04-22T07:06:18.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.210+0000 7f544c992640 1 -- 172.21.15.44:0/3563288542 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f54481589d0 con 0x7f5448067dc0 2024-04-22T07:06:18.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.211+0000 7f5436ffd640 1 -- 172.21.15.44:0/3563288542 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f543807bd00 con 0x7f5448067dc0 2024-04-22T07:06:18.210 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T07:06:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 -- 172.21.15.44:0/3563288542 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f542806c8f0 msgr2=0x7f542806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f542806c8f0 0x7f542806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5448151ad0 tx=0x7f543c008040 comp rx=0 tx=0).stop 2024-04-22T07:06:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 -- 172.21.15.44:0/3563288542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5448067dc0 msgr2=0x7f5448153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5448067dc0 0x7f5448153ae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f543802b9e0 tx=0x7f5438004730 comp rx=0 tx=0).stop 2024-04-22T07:06:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 -- 172.21.15.44:0/3563288542 shutdown_connections 2024-04-22T07:06:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f542806c8f0 0x7f542806edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f54480cb6e0 0x7f54480c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.212 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54480cb300 0x7f5448156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 --2- 172.21.15.44:0/3563288542 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f5448067dc0 0x7f5448153ae0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 -- 172.21.15.44:0/3563288542 >> 172.21.15.44:0/3563288542 conn(0x7f544806fa50 msgr2=0x7f54480c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 -- 172.21.15.44:0/3563288542 shutdown_connections 2024-04-22T07:06:18.213 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.213+0000 7f544c992640 1 -- 172.21.15.44:0/3563288542 wait complete. 2024-04-22T07:06:18.224 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T07:06:18.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.330+0000 7f47c66d2640 1 Processor -- start 2024-04-22T07:06:18.329 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.331+0000 7f47c66d2640 1 -- start start 2024-04-22T07:06:18.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.331+0000 7f47c66d2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c00cd6f0 0x7f47c015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.331+0000 7f47c66d2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f47c00cb3e0 0x7f47c00cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.331+0000 7f47c66d2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c015ee90 con 0x7f47c0154f70 2024-04-22T07:06:18.330 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.331+0000 7f47c66d2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f47c00cd260 con 0x7f47c00cb3e0 2024-04-22T07:06:18.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.331+0000 7f47c66d2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f47c015f170 con 0x7f47c00cd6f0 2024-04-22T07:06:18.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.332+0000 7f47bf7fe640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c00cd6f0 0x7f47c015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.332+0000 7f47c4c6f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f47c00cb3e0 0x7f47c00cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.332+0000 7f47bffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f47c0154f70 0x7f47c00c5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.44:57972/0 (socket says 172.21.15.44:57972) 2024-04-22T07:06:18.331 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.332+0000 7f47bffff640 1 -- 172.21.15.44:0/1379356357 learned_addr learned my addr 172.21.15.44:0/1379356357 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:06:18.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.332+0000 7f47bf7fe640 1 -- 172.21.15.44:0/1379356357 >> v2:172.21.15.70:3300/0 conn(0x7f47c00cb3e0 msgr2=0x7f47c00cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:18.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.332+0000 7f47bf7fe640 1 --2- 172.21.15.44:0/1379356357 >> v2:172.21.15.70:3300/0 conn(0x7f47c00cb3e0 0x7f47c00cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.332+0000 7f47bf7fe640 1 -- 172.21.15.44:0/1379356357 >> v1:172.21.15.70:6789/0 conn(0x7f47c0154f70 legacy=0x7f47c00c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.339 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.332+0000 7f47bf7fe640 1 -- 172.21.15.44:0/1379356357 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c00cc930 con 0x7f47c00cd6f0 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.332+0000 7f47c4c6f640 1 --2- 172.21.15.44:0/1379356357 >> v2:172.21.15.70:3300/0 conn(0x7f47c00cb3e0 0x7f47c00cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.333+0000 7f47bf7fe640 1 --2- 172.21.15.44:0/1379356357 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c00cd6f0 0x7f47c015ad30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47b40097a0 tx=0x7f47b402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5a199cb926d5501 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.333+0000 7f47beffd640 1 -- 172.21.15.44:0/1379356357 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4034070 con 0x7f47c00cd6f0 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.333+0000 7f47beffd640 1 -- 172.21.15.44:0/1379356357 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b4030800 con 0x7f47c00cd6f0 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.333+0000 7f47beffd640 1 -- 172.21.15.44:0/1379356357 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4030bd0 con 0x7f47c00cd6f0 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.334+0000 7f47c66d2640 1 -- 172.21.15.44:0/1379356357 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c00cd6f0 msgr2=0x7f47c015ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.334+0000 7f47c66d2640 1 --2- 172.21.15.44:0/1379356357 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c00cd6f0 0x7f47c015ad30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47b40097a0 tx=0x7f47b402b940 comp rx=0 tx=0).stop 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.334+0000 7f47c66d2640 1 -- 172.21.15.44:0/1379356357 shutdown_connections 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.334+0000 7f47c66d2640 1 --2- 172.21.15.44:0/1379356357 >> v2:172.21.15.70:3300/0 conn(0x7f47c00cb3e0 0x7f47c00cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.334+0000 7f47c66d2640 1 --2- 172.21.15.44:0/1379356357 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c00cd6f0 0x7f47c015ad30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.334+0000 7f47c66d2640 1 -- 172.21.15.44:0/1379356357 >> 172.21.15.44:0/1379356357 conn(0x7f47c0067d40 msgr2=0x7f47c006b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.334+0000 7f47c66d2640 1 -- 172.21.15.44:0/1379356357 shutdown_connections 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.334+0000 7f47c66d2640 1 -- 172.21.15.44:0/1379356357 wait complete. 2024-04-22T07:06:18.341 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.335+0000 7f47c66d2640 1 Processor -- start 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.335+0000 7f47c66d2640 1 -- start start 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47c66d2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c00cb3e0 0x7f47c01f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47c66d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47c00cd6f0 0x7f47c0157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47c66d2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c0154f70 0x7f47c0156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47c66d2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f47c0155350 con 0x7f47c00cb3e0 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47c66d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00ccef0 con 0x7f47c00cd6f0 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47c66d2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f47c014bd30 con 0x7f47c0154f70 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47bffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c00cb3e0 0x7f47c01f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47bffff640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c00cb3e0 0x7f47c01f1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42866/0 (socket says 172.21.15.44:42866) 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47bffff640 1 -- 172.21.15.44:0/41060628 learned_addr learned my addr 172.21.15.44:0/41060628 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47c4c6f640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c0154f70 0x7f47c0156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.336+0000 7f47bf7fe640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47c00cd6f0 0x7f47c0157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.342 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47bffff640 1 -- 172.21.15.44:0/41060628 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c0154f70 msgr2=0x7f47c0156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47bffff640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c0154f70 0x7f47c0156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47bffff640 1 -- 172.21.15.44:0/41060628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47c00cd6f0 msgr2=0x7f47c0157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47bffff640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47c00cd6f0 0x7f47c0157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47bffff640 1 -- 172.21.15.44:0/41060628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c01f1090 con 0x7f47c00cb3e0 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47c4c6f640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c0154f70 0x7f47c0156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47bffff640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c00cb3e0 0x7f47c01f1870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f47b000c3c0 tx=0x7f47b000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47bcff9640 1 -- 172.21.15.44:0/41060628 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0017070 con 0x7f47c00cb3e0 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47bcff9640 1 -- 172.21.15.44:0/41060628 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b0005ce0 con 0x7f47c00cb3e0 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47bcff9640 1 -- 172.21.15.44:0/41060628 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b00072f0 con 0x7f47c00cb3e0 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47c66d2640 1 -- 172.21.15.44:0/41060628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c014bfa0 con 0x7f47c00cb3e0 2024-04-22T07:06:18.343 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.337+0000 7f47c66d2640 1 -- 172.21.15.44:0/41060628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c01f1350 con 0x7f47c00cb3e0 2024-04-22T07:06:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.339+0000 7f47c66d2640 1 -- 172.21.15.44:0/41060628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c01f0ea0 con 0x7f47c00cb3e0 2024-04-22T07:06:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.339+0000 7f47bcff9640 1 -- 172.21.15.44:0/41060628 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47b001a070 con 0x7f47c00cb3e0 2024-04-22T07:06:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.339+0000 7f47bcff9640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.340+0000 7f47bcff9640 1 -- 172.21.15.44:0/41060628 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f47b008aeb0 con 0x7f47c00cb3e0 2024-04-22T07:06:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.340+0000 7f47bf7fe640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.344 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.340+0000 7f47bf7fe640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f47ac06c8f0 0x7f47ac06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f47b4036000 tx=0x7f47b403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.346 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.347+0000 7f47bcff9640 1 -- 172.21.15.44:0/41060628 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f47b0058b30 con 0x7f47c00cb3e0 2024-04-22T07:06:18.544 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.546+0000 7f47c66d2640 1 -- 172.21.15.44:0/41060628 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47c01f1520 con 0x7f47c00cb3e0 2024-04-22T07:06:18.545 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.546+0000 7f47bcff9640 1 -- 172.21.15.44:0/41060628 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v26) ==== 80+0+122 (secure 0 0 0) 0x7f47b005ce30 con 0x7f47c00cb3e0 2024-04-22T07:06:18.545 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.70:6834/1969018412 2024-04-23T07:05:49.227539+0000 2024-04-22T07:06:18.545 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.70:6832/1969018412 2024-04-23T07:05:49.227539+0000 2024-04-22T07:06:18.545 INFO:teuthology.orchestra.run.smithi044.stderr:listed 2 entries 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 -- 172.21.15.44:0/41060628 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f47ac06c8f0 msgr2=0x7f47ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f47ac06c8f0 0x7f47ac06edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f47b4036000 tx=0x7f47b403b040 comp rx=0 tx=0).stop 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 -- 172.21.15.44:0/41060628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c00cb3e0 msgr2=0x7f47c01f1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c00cb3e0 0x7f47c01f1870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f47b000c3c0 tx=0x7f47b000c7c0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 -- 172.21.15.44:0/41060628 shutdown_connections 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f47ac06c8f0 0x7f47ac06edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f47c0154f70 0x7f47c0156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47c00cd6f0 0x7f47c0157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 --2- 172.21.15.44:0/41060628 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f47c00cb3e0 0x7f47c01f1870 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 -- 172.21.15.44:0/41060628 >> 172.21.15.44:0/41060628 conn(0x7f47c0067d40 msgr2=0x7f47c00cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 -- 172.21.15.44:0/41060628 shutdown_connections 2024-04-22T07:06:18.548 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.549+0000 7f47c66d2640 1 -- 172.21.15.44:0/41060628 wait complete. 2024-04-22T07:06:18.559 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:06:18.559 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T07:06:18.575 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T07:06:18.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.725+0000 7ff4ccfa4640 1 Processor -- start 2024-04-22T07:06:18.724 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.726+0000 7ff4ccfa4640 1 -- start start 2024-04-22T07:06:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.726+0000 7ff4ccfa4640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7ff4c8154ef0 0x7ff4c80ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.726+0000 7ff4ccfa4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c80c9c20 0x7ff4c80cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4ccfa4640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c81552d0 con 0x7ff4c80cb850 2024-04-22T07:06:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4ccfa4640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c80c9360 con 0x7ff4c8154ef0 2024-04-22T07:06:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4ccfa4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c80708a0 con 0x7ff4c80c9c20 2024-04-22T07:06:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4c6d9d640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7ff4c80cb850 0x7ff4c80ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:32796/0 (socket says 172.21.15.44:32796) 2024-04-22T07:06:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4c6d9d640 1 -- 172.21.15.44:0/3232236763 learned_addr learned my addr 172.21.15.44:0/3232236763 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:06:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4c659c640 1 --2- 172.21.15.44:0/3232236763 >> v2:172.21.15.44:3300/0 conn(0x7ff4c8154ef0 0x7ff4c80ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.726 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4c5d9b640 1 --2- 172.21.15.44:0/3232236763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c80c9c20 0x7ff4c80cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4c659c640 1 -- 172.21.15.44:0/3232236763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c80c9c20 msgr2=0x7ff4c80cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:18.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4c559a640 1 -- 172.21.15.44:0/3232236763 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1672970322 0 0) 0x7ff4c81552d0 con 0x7ff4c80cb850 2024-04-22T07:06:18.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4c659c640 1 --2- 172.21.15.44:0/3232236763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c80c9c20 0x7ff4c80cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4c659c640 1 -- 172.21.15.44:0/3232236763 >> v1:172.21.15.44:6789/0 conn(0x7ff4c80cb850 legacy=0x7ff4c80ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.727 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.727+0000 7ff4c659c640 1 -- 172.21.15.44:0/3232236763 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c80c9080 con 0x7ff4c8154ef0 2024-04-22T07:06:18.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.728+0000 7ff4c659c640 1 --2- 172.21.15.44:0/3232236763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c8154ef0 0x7ff4c80ca4e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009a00 tx=0x7ff4b802baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39c5a7a6d82a2ee server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.729+0000 7ff4c559a640 1 -- 172.21.15.44:0/3232236763 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b802d650 con 0x7ff4c8154ef0 2024-04-22T07:06:18.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.729+0000 7ff4c559a640 1 -- 172.21.15.44:0/3232236763 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b802d7f0 con 0x7ff4c8154ef0 2024-04-22T07:06:18.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.729+0000 7ff4c559a640 1 -- 172.21.15.44:0/3232236763 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b802dbc0 con 0x7ff4c8154ef0 2024-04-22T07:06:18.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.729+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/3232236763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c8154ef0 msgr2=0x7ff4c80ca4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.728 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.729+0000 7ff4ccfa4640 1 --2- 172.21.15.44:0/3232236763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c8154ef0 0x7ff4c80ca4e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009a00 tx=0x7ff4b802baa0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.729+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/3232236763 shutdown_connections 2024-04-22T07:06:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.729+0000 7ff4ccfa4640 1 --2- 172.21.15.44:0/3232236763 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c80c9c20 0x7ff4c80cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.729+0000 7ff4ccfa4640 1 --2- 172.21.15.44:0/3232236763 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c8154ef0 0x7ff4c80ca4e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.729+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/3232236763 >> 172.21.15.44:0/3232236763 conn(0x7ff4c8071850 msgr2=0x7ff4c8071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.730+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/3232236763 shutdown_connections 2024-04-22T07:06:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.730+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/3232236763 wait complete. 2024-04-22T07:06:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.731+0000 7ff4ccfa4640 1 Processor -- start 2024-04-22T07:06:18.729 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.731+0000 7ff4ccfa4640 1 -- start start 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4ccfa4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c80c9c20 0x7ff4c80c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4ccfa4640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c80cb850 0x7ff4c81f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4ccfa4640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c8154ef0 0x7ff4c8153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4ccfa4640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80705c0 con 0x7ff4c80cb850 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4ccfa4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80702e0 con 0x7ff4c80c9c20 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4ccfa4640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c815e1c0 con 0x7ff4c8154ef0 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c80cb850 0x7ff4c81f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c5d9b640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c80cb850 0x7ff4c81f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42882/0 (socket says 172.21.15.44:42882) 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c5d9b640 1 -- 172.21.15.44:0/870992986 learned_addr learned my addr 172.21.15.44:0/870992986 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c659c640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c80c9c20 0x7ff4c80c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.732 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c6d9d640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c8154ef0 0x7ff4c8153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c5d9b640 1 -- 172.21.15.44:0/870992986 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c8154ef0 msgr2=0x7ff4c8153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c5d9b640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c8154ef0 0x7ff4c8153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c5d9b640 1 -- 172.21.15.44:0/870992986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c80c9c20 msgr2=0x7ff4c80c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c5d9b640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c80c9c20 0x7ff4c80c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c5d9b640 1 -- 172.21.15.44:0/870992986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c81f0ea0 con 0x7ff4c80cb850 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.732+0000 7ff4c6d9d640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c8154ef0 0x7ff4c8153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.733+0000 7ff4c5d9b640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c80cb850 0x7ff4c81f1960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00e8a0 tx=0x7ff4bc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.733+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/870992986 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc017070 con 0x7ff4c80cb850 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.733+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/870992986 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4bc00c940 con 0x7ff4c80cb850 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.733+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/870992986 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00cd30 con 0x7ff4c80cb850 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.733+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/870992986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c80ce8c0 con 0x7ff4c80cb850 2024-04-22T07:06:18.733 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.733+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/870992986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c80cee80 con 0x7ff4c80cb850 2024-04-22T07:06:18.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.734+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/870992986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c80c4d90 con 0x7ff4c80cb850 2024-04-22T07:06:18.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.735+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/870992986 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4bc01a070 con 0x7ff4c80cb850 2024-04-22T07:06:18.734 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.736+0000 7ff4b6ffd640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4a806c8f0 0x7ff4a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:18.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.736+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/870992986 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7ff4bc08bb90 con 0x7ff4c80cb850 2024-04-22T07:06:18.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.736+0000 7ff4c659c640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4a806c8f0 0x7ff4a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:18.735 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.737+0000 7ff4c659c640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4a806c8f0 0x7ff4a806edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8036000 tx=0x7ff4b8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:18.743 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.740+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/870992986 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4bc059e50 con 0x7ff4c80cb850 2024-04-22T07:06:18.928 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.929+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/870992986 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4c81585f0 con 0x7ff4c80cb850 2024-04-22T07:06:18.929 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.931+0000 7ff4b6ffd640 1 -- 172.21.15.44:0/870992986 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+16936 (secure 0 0 0) 0x7ff4bc059750 con 0x7ff4c80cb850 2024-04-22T07:06:18.930 INFO:teuthology.orchestra.run.smithi044.stdout: 2024-04-22T07:06:18.930 INFO:teuthology.orchestra.run.smithi044.stdout:{"epoch":26,"fsid":"dc21e26c-6c65-477f-a367-ffb001b78b47","created":"2024-04-22T07:03:49.432140+0000","modified":"2024-04-22T07:05:49.227595+0000","last_up_change":"2024-04-22T07:04:00.126646+0000","last_in_change":"2024-04-22T07:03:51.577561+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T07:04:01.723984+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T07:04:11.652423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T07:04:12.098854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b02b7a83-3c8d-4357-b2cb-4d0a5868a452","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6824","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6825","nonce":1806128924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6826","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6827","nonce":1806128924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6830","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6831","nonce":1806128924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6828","nonce":1806128924},{"type":"v1","addr":"172.21.15.44:6829","nonce":1806128924}]},"public_addr":"172.21.15.44:6825/1806128924","cluster_addr":"172.21.15.44:6827/1806128924","heartbeat_back_addr":"172.21.15.44:6831/1806128924","heartbeat_front_addr":"172.21.15.44:6829/1806128924","state":["exists","up"]},{"osd":1,"uuid":"d59d97eb-4bd1-4884-bd1a-a028ee23283f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6808","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6809","nonce":2837839457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6810","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6811","nonce":2837839457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6814","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6815","nonce":2837839457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6812","nonce":2837839457},{"type":"v1","addr":"172.21.15.44:6813","nonce":2837839457}]},"public_addr":"172.21.15.44:6809/2837839457","cluster_addr":"172.21.15.44:6811/2837839457","heartbeat_back_addr":"172.21.15.44:6815/2837839457","heartbeat_front_addr":"172.21.15.44:6813/2837839457","state":["exists","up"]},{"osd":2,"uuid":"df8331b0-946c-4b09-8cab-af825d9a1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6800","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6801","nonce":3148242873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6802","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6803","nonce":3148242873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6806","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6807","nonce":3148242873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6804","nonce":3148242873},{"type":"v1","addr":"172.21.15.44:6805","nonce":3148242873}]},"public_addr":"172.21.15.44:6801/3148242873","cluster_addr":"172.21.15.44:6803/3148242873","heartbeat_back_addr":"172.21.15.44:6807/3148242873","heartbeat_front_addr":"172.21.15.44:6805/3148242873","state":["exists","up"]},{"osd":3,"uuid":"a2849b7d-4d44-4107-a97f-f24c6b94ee4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6816","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6817","nonce":2173398368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6818","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6819","nonce":2173398368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6822","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6823","nonce":2173398368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.44:6820","nonce":2173398368},{"type":"v1","addr":"172.21.15.44:6821","nonce":2173398368}]},"public_addr":"172.21.15.44:6817/2173398368","cluster_addr":"172.21.15.44:6819/2173398368","heartbeat_back_addr":"172.21.15.44:6823/2173398368","heartbeat_front_addr":"172.21.15.44:6821/2173398368","state":["exists","up"]},{"osd":4,"uuid":"e6d3626b-9227-40fc-af5d-ff49b5da155f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6818","nonce":1770474213}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6821","nonce":1770474213}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6827","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6828","nonce":1770474213}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":1770474213},{"type":"v1","addr":"172.21.15.70:6825","nonce":1770474213}]},"public_addr":"172.21.15.70:6818/1770474213","cluster_addr":"172.21.15.70:6821/1770474213","heartbeat_back_addr":"172.21.15.70:6828/1770474213","heartbeat_front_addr":"172.21.15.70:6825/1770474213","state":["exists","up"]},{"osd":5,"uuid":"27e16c28-3d8e-451c-8c1f-1a60b1619f6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6819","nonce":1512457027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6824","nonce":1512457027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6831","nonce":1512457027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1512457027},{"type":"v1","addr":"172.21.15.70:6829","nonce":1512457027}]},"public_addr":"172.21.15.70:6819/1512457027","cluster_addr":"172.21.15.70:6824/1512457027","heartbeat_back_addr":"172.21.15.70:6831/1512457027","heartbeat_front_addr":"172.21.15.70:6829/1512457027","state":["exists","up"]},{"osd":6,"uuid":"91d9b4da-a081-4f7b-9e3b-34cf81c301bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6801","nonce":921795404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6803","nonce":921795404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6807","nonce":921795404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":921795404},{"type":"v1","addr":"172.21.15.70:6805","nonce":921795404}]},"public_addr":"172.21.15.70:6801/921795404","cluster_addr":"172.21.15.70:6803/921795404","heartbeat_back_addr":"172.21.15.70:6807/921795404","heartbeat_front_addr":"172.21.15.70:6805/921795404","state":["exists","up"]},{"osd":7,"uuid":"cec9c5a6-96c3-4a2e-8e53-bc69804ff685","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6809","nonce":3781953159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6811","nonce":3781953159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6815","nonce":3781953159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3781953159},{"type":"v1","addr":"172.21.15.70:6813","nonce":3781953159}]},"public_addr":"172.21.15.70:6809/3781953159","cluster_addr":"172.21.15.70:6811/3781953159","heartbeat_back_addr":"172.21.15.70:6815/3781953159","heartbeat_front_addr":"172.21.15.70:6813/3781953159","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.722640+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.741922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.648905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.776063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.790948+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.773173+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.749350+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T07:03:56.716584+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6834/1969018412":"2024-04-23T07:05:49.227539+0000","172.21.15.70:6832/1969018412":"2024-04-23T07:05:49.227539+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T07:06:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.933+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/870992986 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4a806c8f0 msgr2=0x7ff4a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.933+0000 7ff4ccfa4640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4a806c8f0 0x7ff4a806edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8036000 tx=0x7ff4b8008030 comp rx=0 tx=0).stop 2024-04-22T07:06:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.933+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/870992986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c80cb850 msgr2=0x7ff4c81f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.933+0000 7ff4ccfa4640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c80cb850 0x7ff4c81f1960 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00e8a0 tx=0x7ff4bc00ed60 comp rx=0 tx=0).stop 2024-04-22T07:06:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.933+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/870992986 shutdown_connections 2024-04-22T07:06:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.933+0000 7ff4ccfa4640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7ff4a806c8f0 0x7ff4a806edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.933+0000 7ff4ccfa4640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7ff4c8154ef0 0x7ff4c8153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.933+0000 7ff4ccfa4640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7ff4c80cb850 0x7ff4c81f1960 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.933+0000 7ff4ccfa4640 1 --2- 172.21.15.44:0/870992986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4c80c9c20 0x7ff4c80c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:18.932 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.933+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/870992986 >> 172.21.15.44:0/870992986 conn(0x7ff4c8071850 msgr2=0x7ff4c80c32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:18.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.934+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/870992986 shutdown_connections 2024-04-22T07:06:18.933 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:18.934+0000 7ff4ccfa4640 1 -- 172.21.15.44:0/870992986 wait complete. 2024-04-22T07:06:18.944 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T07:04:01.723984+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T07:04:11.652423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T07:04:12.098854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T07:06:18.944 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T07:06:19.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.050+0000 7f73540d2640 1 Processor -- start 2024-04-22T07:06:19.049 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.050+0000 7f73540d2640 1 -- start start 2024-04-22T07:06:19.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.051+0000 7f73540d2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c152d90 0x7f734c15c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.051+0000 7f73540d2640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f734c0ca1a0 0x7f734c071300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.051+0000 7f73540d2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f734c0c9c50 0x7f734c071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.050 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.051+0000 7f73540d2640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f734c0c54c0 con 0x7f734c0ca1a0 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.051+0000 7f73540d2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f734c0cb2a0 con 0x7f734c0c9c50 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.051+0000 7f73540d2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f734c0c9690 con 0x7f734c152d90 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.051+0000 7f735166d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f734c0ca1a0 0x7f734c071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.051+0000 7f735166d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f734c0ca1a0 0x7f734c071300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42892/0 (socket says 172.21.15.44:42892) 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.051+0000 7f735166d640 1 -- 172.21.15.44:0/1089482540 learned_addr learned my addr 172.21.15.44:0/1089482540 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.052+0000 7f735266f640 1 --2- 172.21.15.44:0/1089482540 >> v2:172.21.15.70:3300/0 conn(0x7f734c0c9c50 0x7f734c071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.052+0000 7f7351e6e640 1 --2- 172.21.15.44:0/1089482540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c152d90 0x7f734c15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.052+0000 7f735166d640 1 -- 172.21.15.44:0/1089482540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c152d90 msgr2=0x7f734c15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.052+0000 7f735166d640 1 --2- 172.21.15.44:0/1089482540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c152d90 0x7f734c15c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.052+0000 7f735166d640 1 -- 172.21.15.44:0/1089482540 >> v2:172.21.15.70:3300/0 conn(0x7f734c0c9c50 msgr2=0x7f734c071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.052+0000 7f735166d640 1 --2- 172.21.15.44:0/1089482540 >> v2:172.21.15.70:3300/0 conn(0x7f734c0c9c50 0x7f734c071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.052+0000 7f735166d640 1 -- 172.21.15.44:0/1089482540 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c0ca780 con 0x7f734c0ca1a0 2024-04-22T07:06:19.051 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.052+0000 7f735266f640 1 --2- 172.21.15.44:0/1089482540 >> v2:172.21.15.70:3300/0 conn(0x7f734c0c9c50 0x7f734c071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:06:19.052 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.052+0000 7f735166d640 1 --2- 172.21.15.44:0/1089482540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 0x7f734c071300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f734c062cb0 tx=0x7f733c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6bed280f8f0b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.053+0000 7f7350e6c640 1 -- 172.21.15.44:0/1089482540 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c034070 con 0x7f734c0ca1a0 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.053+0000 7f7350e6c640 1 -- 172.21.15.44:0/1089482540 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733c030850 con 0x7f734c0ca1a0 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.053+0000 7f7350e6c640 1 -- 172.21.15.44:0/1089482540 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c030c20 con 0x7f734c0ca1a0 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.053+0000 7f73540d2640 1 -- 172.21.15.44:0/1089482540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 msgr2=0x7f734c071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.053+0000 7f73540d2640 1 --2- 172.21.15.44:0/1089482540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 0x7f734c071300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f734c062cb0 tx=0x7f733c02b990 comp rx=0 tx=0).stop 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.053+0000 7f73540d2640 1 -- 172.21.15.44:0/1089482540 shutdown_connections 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.053+0000 7f73540d2640 1 --2- 172.21.15.44:0/1089482540 >> v2:172.21.15.70:3300/0 conn(0x7f734c0c9c50 0x7f734c071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.053+0000 7f73540d2640 1 --2- 172.21.15.44:0/1089482540 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 0x7f734c071300 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.053+0000 7f73540d2640 1 --2- 172.21.15.44:0/1089482540 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c152d90 0x7f734c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.053+0000 7f73540d2640 1 -- 172.21.15.44:0/1089482540 >> 172.21.15.44:0/1089482540 conn(0x7f734c06fa50 msgr2=0x7f734c0676b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.054+0000 7f73540d2640 1 -- 172.21.15.44:0/1089482540 shutdown_connections 2024-04-22T07:06:19.053 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.054+0000 7f73540d2640 1 -- 172.21.15.44:0/1089482540 wait complete. 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.055+0000 7f73540d2640 1 Processor -- start 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.055+0000 7f73540d2640 1 -- start start 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.055+0000 7f73540d2640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c0c9c50 0x7f734c141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.055+0000 7f73540d2640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 0x7f734c147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.055+0000 7f73540d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c152d90 0x7f734c147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f73540d2640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f734c153170 con 0x7f734c0ca1a0 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f73540d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f734c0c9970 con 0x7f734c152d90 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f73540d2640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f734c0cb930 con 0x7f734c0c9c50 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735166d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 0x7f734c147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735166d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 0x7f734c147450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42904/0 (socket says 172.21.15.44:42904) 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735166d640 1 -- 172.21.15.44:0/705412609 learned_addr learned my addr 172.21.15.44:0/705412609 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f7351e6e640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c0c9c50 0x7f734c141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.056 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735266f640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c152d90 0x7f734c147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735166d640 1 -- 172.21.15.44:0/705412609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c0c9c50 msgr2=0x7f734c141c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735166d640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c0c9c50 0x7f734c141c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735166d640 1 -- 172.21.15.44:0/705412609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c152d90 msgr2=0x7f734c147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735166d640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c152d90 0x7f734c147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735166d640 1 -- 172.21.15.44:0/705412609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c144d40 con 0x7f734c0ca1a0 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735266f640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c152d90 0x7f734c147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f7351e6e640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c0c9c50 0x7f734c141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f735166d640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 0x7f734c147450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f733c02b9d0 tx=0x7f733c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.056+0000 7f73427fc640 1 -- 172.21.15.44:0/705412609 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c048070 con 0x7f734c0ca1a0 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.057+0000 7f73427fc640 1 -- 172.21.15.44:0/705412609 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733c030650 con 0x7f734c0ca1a0 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.057+0000 7f73540d2640 1 -- 172.21.15.44:0/705412609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c144fe0 con 0x7f734c0ca1a0 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.057+0000 7f73540d2640 1 -- 172.21.15.44:0/705412609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c1455a0 con 0x7f734c0ca1a0 2024-04-22T07:06:19.057 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.057+0000 7f73427fc640 1 -- 172.21.15.44:0/705412609 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c034040 con 0x7f734c0ca1a0 2024-04-22T07:06:19.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.058+0000 7f73540d2640 1 -- 172.21.15.44:0/705412609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7334005290 con 0x7f734c0ca1a0 2024-04-22T07:06:19.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.059+0000 7f73427fc640 1 -- 172.21.15.44:0/705412609 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f733c04b070 con 0x7f734c0ca1a0 2024-04-22T07:06:19.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.059+0000 7f73427fc640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f733006c920 0x7f733006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.058 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.060+0000 7f7351e6e640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f733006c920 0x7f733006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.060+0000 7f73427fc640 1 -- 172.21.15.44:0/705412609 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f733c0add60 con 0x7f734c0ca1a0 2024-04-22T07:06:19.059 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.060+0000 7f7351e6e640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f733006c920 0x7f733006ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7348006fd0 tx=0x7f734800f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:19.066 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.068+0000 7f73427fc640 1 -- 172.21.15.44:0/705412609 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f733c07c020 con 0x7f734c0ca1a0 2024-04-22T07:06:19.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.276+0000 7f73540d2640 1 -- 172.21.15.44:0/705412609 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7334005d90 con 0x7f734c0ca1a0 2024-04-22T07:06:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.277+0000 7f73427fc640 1 -- 172.21.15.44:0/705412609 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f733c07b920 con 0x7f734c0ca1a0 2024-04-22T07:06:19.276 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 1 2024-04-22T07:06:19.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.279+0000 7f73540d2640 1 -- 172.21.15.44:0/705412609 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f733006c920 msgr2=0x7f733006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f733006c920 0x7f733006ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7348006fd0 tx=0x7f734800f040 comp rx=0 tx=0).stop 2024-04-22T07:06:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 -- 172.21.15.44:0/705412609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 msgr2=0x7f734c147450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 0x7f734c147450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f733c02b9d0 tx=0x7f733c004730 comp rx=0 tx=0).stop 2024-04-22T07:06:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 -- 172.21.15.44:0/705412609 shutdown_connections 2024-04-22T07:06:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f733006c920 0x7f733006ede0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c152d90 0x7f734c147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f734c0ca1a0 0x7f734c147450 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 --2- 172.21.15.44:0/705412609 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f734c0c9c50 0x7f734c141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.279 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 -- 172.21.15.44:0/705412609 >> 172.21.15.44:0/705412609 conn(0x7f734c06fa50 msgr2=0x7f734c0c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 -- 172.21.15.44:0/705412609 shutdown_connections 2024-04-22T07:06:19.280 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.280+0000 7f73540d2640 1 -- 172.21.15.44:0/705412609 wait complete. 2024-04-22T07:06:19.293 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T07:06:19.453 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.454+0000 7f2513d23640 1 Processor -- start 2024-04-22T07:06:19.454 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.455+0000 7f2513d23640 1 -- start start 2024-04-22T07:06:19.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.456+0000 7f2513d23640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f250c154e30 0x7f250c0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.456+0000 7f2513d23640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f250c0ced50 0x7f250c165560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.456+0000 7f2513d23640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f250c15b2f0 con 0x7f250c154a50 2024-04-22T07:06:19.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.456+0000 7f2513d23640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f250c06ff40 con 0x7f250c0ced50 2024-04-22T07:06:19.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.456+0000 7f2513d23640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f250c0c96a0 con 0x7f250c154e30 2024-04-22T07:06:19.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.456+0000 7f25122c0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f250c0ced50 0x7f250c165560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.456+0000 7f25122c0640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f250c0ced50 0x7f250c165560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42916/0 (socket says 172.21.15.44:42916) 2024-04-22T07:06:19.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.456+0000 7f25122c0640 1 -- 172.21.15.44:0/2851223086 learned_addr learned my addr 172.21.15.44:0/2851223086 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:19.455 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.456+0000 7f25112be640 1 --2- 172.21.15.44:0/2851223086 >> v2:172.21.15.70:3300/0 conn(0x7f250c154e30 0x7f250c0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.457+0000 7f25122c0640 1 -- 172.21.15.44:0/2851223086 >> v2:172.21.15.70:3300/0 conn(0x7f250c154e30 msgr2=0x7f250c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.457+0000 7f25122c0640 1 --2- 172.21.15.44:0/2851223086 >> v2:172.21.15.70:3300/0 conn(0x7f250c154e30 0x7f250c0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.457+0000 7f25122c0640 1 -- 172.21.15.44:0/2851223086 >> v1:172.21.15.44:6789/0 conn(0x7f250c154a50 legacy=0x7f250c15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.456 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.457+0000 7f25122c0640 1 -- 172.21.15.44:0/2851223086 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c0c90e0 con 0x7f250c0ced50 2024-04-22T07:06:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.458+0000 7f25122c0640 1 --2- 172.21.15.44:0/2851223086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0ced50 0x7f250c165560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f250c06d1c0 tx=0x7f250402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808dd3633c2d1306 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.458+0000 7f2510abd640 1 -- 172.21.15.44:0/2851223086 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250402c8d0 con 0x7f250c0ced50 2024-04-22T07:06:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.458+0000 7f2510abd640 1 -- 172.21.15.44:0/2851223086 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250402ca70 con 0x7f250c0ced50 2024-04-22T07:06:19.457 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.458+0000 7f2510abd640 1 -- 172.21.15.44:0/2851223086 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250402ce40 con 0x7f250c0ced50 2024-04-22T07:06:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.459+0000 7f2513d23640 1 -- 172.21.15.44:0/2851223086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0ced50 msgr2=0x7f250c165560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.459+0000 7f2513d23640 1 --2- 172.21.15.44:0/2851223086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0ced50 0x7f250c165560 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f250c06d1c0 tx=0x7f250402bdb0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.459+0000 7f2513d23640 1 -- 172.21.15.44:0/2851223086 shutdown_connections 2024-04-22T07:06:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.459+0000 7f2513d23640 1 --2- 172.21.15.44:0/2851223086 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c0ced50 0x7f250c165560 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.459+0000 7f2513d23640 1 --2- 172.21.15.44:0/2851223086 >> v2:172.21.15.70:3300/0 conn(0x7f250c154e30 0x7f250c0ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.459+0000 7f2513d23640 1 -- 172.21.15.44:0/2851223086 >> 172.21.15.44:0/2851223086 conn(0x7f250c0718a0 msgr2=0x7f250c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.459+0000 7f2513d23640 1 -- 172.21.15.44:0/2851223086 shutdown_connections 2024-04-22T07:06:19.458 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.459+0000 7f2513d23640 1 -- 172.21.15.44:0/2851223086 wait complete. 2024-04-22T07:06:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.461+0000 7f2513d23640 1 Processor -- start 2024-04-22T07:06:19.459 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.461+0000 7f2513d23640 1 -- start start 2024-04-22T07:06:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f2513d23640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f250c0ced50 0x7f250c150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f2513d23640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f250c154a50 0x7f250c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f2513d23640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c154e30 0x7f250c146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f2513d23640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f250c0c9c20 con 0x7f250c154e30 2024-04-22T07:06:19.462 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f2513d23640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f250c0c9940 con 0x7f250c0ced50 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f2513d23640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f250c070470 con 0x7f250c154a50 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f25122c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c154e30 0x7f250c146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f25122c0640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c154e30 0x7f250c146ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42928/0 (socket says 172.21.15.44:42928) 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f25122c0640 1 -- 172.21.15.44:0/457128615 learned_addr learned my addr 172.21.15.44:0/457128615 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f25112be640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f250c154a50 0x7f250c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.462+0000 7f2511abf640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f250c0ced50 0x7f250c150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f25122c0640 1 -- 172.21.15.44:0/457128615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f250c154a50 msgr2=0x7f250c151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f25122c0640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f250c154a50 0x7f250c151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f25122c0640 1 -- 172.21.15.44:0/457128615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f250c0ced50 msgr2=0x7f250c150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f25122c0640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f250c0ced50 0x7f250c150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f25122c0640 1 -- 172.21.15.44:0/457128615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c06cef0 con 0x7f250c154e30 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f2511abf640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f250c0ced50 0x7f250c150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f25122c0640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c154e30 0x7f250c146ee0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2504009f40 tx=0x7f2504004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:19.463 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f25027fc640 1 -- 172.21.15.44:0/457128615 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504043020 con 0x7f250c154e30 2024-04-22T07:06:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f25027fc640 1 -- 172.21.15.44:0/457128615 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250403e450 con 0x7f250c154e30 2024-04-22T07:06:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f2513d23640 1 -- 172.21.15.44:0/457128615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c0c5bb0 con 0x7f250c154e30 2024-04-22T07:06:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f25027fc640 1 -- 172.21.15.44:0/457128615 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504009510 con 0x7f250c154e30 2024-04-22T07:06:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.463+0000 7f2513d23640 1 -- 172.21.15.44:0/457128615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c0c6170 con 0x7f250c154e30 2024-04-22T07:06:19.464 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.465+0000 7f2513d23640 1 -- 172.21.15.44:0/457128615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250c0cd4f0 con 0x7f250c154e30 2024-04-22T07:06:19.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.465+0000 7f25027fc640 1 -- 172.21.15.44:0/457128615 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2504046020 con 0x7f250c154e30 2024-04-22T07:06:19.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.465+0000 7f25027fc640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f24f006c920 0x7f24f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.465+0000 7f25027fc640 1 -- 172.21.15.44:0/457128615 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f25040ae0a0 con 0x7f250c154e30 2024-04-22T07:06:19.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.466+0000 7f2511abf640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f24f006c920 0x7f24f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.466+0000 7f2511abf640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f24f006c920 0x7f24f006ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2508000f60 tx=0x7f2508009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:19.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.470+0000 7f25027fc640 1 -- 172.21.15.44:0/457128615 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f250c0cd4f0 con 0x7f250c154e30 2024-04-22T07:06:19.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.847+0000 7f2513d23640 1 -- 172.21.15.44:0/457128615 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f250c069210 con 0x7f250c154e30 2024-04-22T07:06:19.846 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.847+0000 7f25027fc640 1 -- 172.21.15.44:0/457128615 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f250404d070 con 0x7f250c154e30 2024-04-22T07:06:19.846 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 32 2024-04-22T07:06:19.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 -- 172.21.15.44:0/457128615 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f24f006c920 msgr2=0x7f24f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f24f006c920 0x7f24f006ede0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2508000f60 tx=0x7f2508009040 comp rx=0 tx=0).stop 2024-04-22T07:06:19.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 -- 172.21.15.44:0/457128615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c154e30 msgr2=0x7f250c146ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.849 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c154e30 0x7f250c146ee0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2504009f40 tx=0x7f2504004a40 comp rx=0 tx=0).stop 2024-04-22T07:06:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 -- 172.21.15.44:0/457128615 shutdown_connections 2024-04-22T07:06:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f24f006c920 0x7f24f006ede0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f250c154a50 0x7f250c151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f250c154e30 0x7f250c146ee0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 --2- 172.21.15.44:0/457128615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f250c0ced50 0x7f250c150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 -- 172.21.15.44:0/457128615 >> 172.21.15.44:0/457128615 conn(0x7f250c0718a0 msgr2=0x7f250c0cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 -- 172.21.15.44:0/457128615 shutdown_connections 2024-04-22T07:06:19.850 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.850+0000 7f2513d23640 1 -- 172.21.15.44:0/457128615 wait complete. 2024-04-22T07:06:19.864 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T07:06:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.971+0000 7f37e0900640 1 Processor -- start 2024-04-22T07:06:19.970 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.971+0000 7f37e0900640 1 -- start start 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.972+0000 7f37e0900640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f37dc1552c0 0x7f37dc0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.972+0000 7f37e0900640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f37dc0caec0 0x7f37dc0d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.972+0000 7f37e0900640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37dc1556a0 con 0x7f37dc0cc920 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.972+0000 7f37e0900640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f37dc15b160 con 0x7f37dc0caec0 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.972+0000 7f37e0900640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f37dc15b440 con 0x7f37dc1552c0 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.972+0000 7f37d9d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f37dc0caec0 0x7f37dc0d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.972+0000 7f37d9d9b640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f37dc0caec0 0x7f37dc0d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42936/0 (socket says 172.21.15.44:42936) 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.972+0000 7f37d9d9b640 1 -- 172.21.15.44:0/2104801210 learned_addr learned my addr 172.21.15.44:0/2104801210 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.972+0000 7f37da59c640 1 --2- 172.21.15.44:0/2104801210 >> v2:172.21.15.70:3300/0 conn(0x7f37dc1552c0 0x7f37dc0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.973+0000 7f37d959a640 1 -- 172.21.15.44:0/2104801210 <== mon.1 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2344085488 0 0) 0x7f37dc1556a0 con 0x7f37dc0cc920 2024-04-22T07:06:19.972 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.973+0000 7f37d9d9b640 1 -- 172.21.15.44:0/2104801210 >> v2:172.21.15.70:3300/0 conn(0x7f37dc1552c0 msgr2=0x7f37dc0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.973+0000 7f37d9d9b640 1 --2- 172.21.15.44:0/2104801210 >> v2:172.21.15.70:3300/0 conn(0x7f37dc1552c0 0x7f37dc0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.973+0000 7f37d9d9b640 1 -- 172.21.15.44:0/2104801210 >> v1:172.21.15.70:6789/0 conn(0x7f37dc0cc920 legacy=0x7f37dc0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.973+0000 7f37d9d9b640 1 -- 172.21.15.44:0/2104801210 --> v2:172.21.15.44:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37dc15b720 con 0x7f37dc0caec0 2024-04-22T07:06:19.973 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.973+0000 7f37da59c640 1 --2- 172.21.15.44:0/2104801210 >> v2:172.21.15.70:3300/0 conn(0x7f37dc1552c0 0x7f37dc0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T07:06:19.974 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.973+0000 7f37d9d9b640 1 --2- 172.21.15.44:0/2104801210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc0caec0 0x7f37dc0d0a20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0d09f0 tx=0x7f37d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6249cdbfada024eb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.974+0000 7f37d959a640 1 -- 172.21.15.44:0/2104801210 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0034070 con 0x7f37dc0caec0 2024-04-22T07:06:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.974+0000 7f37d959a640 1 -- 172.21.15.44:0/2104801210 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d0030860 con 0x7f37dc0caec0 2024-04-22T07:06:19.975 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.974+0000 7f37d959a640 1 -- 172.21.15.44:0/2104801210 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0030c30 con 0x7f37dc0caec0 2024-04-22T07:06:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.974+0000 7f37e0900640 1 -- 172.21.15.44:0/2104801210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc0caec0 msgr2=0x7f37dc0d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.976 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.974+0000 7f37e0900640 1 --2- 172.21.15.44:0/2104801210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc0caec0 0x7f37dc0d0a20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0d09f0 tx=0x7f37d002b9a0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.976+0000 7f37e0900640 1 -- 172.21.15.44:0/2104801210 shutdown_connections 2024-04-22T07:06:19.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.976+0000 7f37e0900640 1 --2- 172.21.15.44:0/2104801210 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc0caec0 0x7f37dc0d0a20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.976+0000 7f37e0900640 1 --2- 172.21.15.44:0/2104801210 >> v2:172.21.15.70:3300/0 conn(0x7f37dc1552c0 0x7f37dc0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.976+0000 7f37e0900640 1 -- 172.21.15.44:0/2104801210 >> 172.21.15.44:0/2104801210 conn(0x7f37dc0676b0 msgr2=0x7f37dc067ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:19.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.978+0000 7f37e0900640 1 -- 172.21.15.44:0/2104801210 shutdown_connections 2024-04-22T07:06:19.977 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.978+0000 7f37e0900640 1 -- 172.21.15.44:0/2104801210 wait complete. 2024-04-22T07:06:19.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.979+0000 7f37e0900640 1 Processor -- start 2024-04-22T07:06:19.978 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.980+0000 7f37e0900640 1 -- start start 2024-04-22T07:06:19.984 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37e0900640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37dc0caec0 0x7f37dc1f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37e0900640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37dc0cc920 0x7f37dc152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37e0900640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc1552c0 0x7f37dc152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37e0900640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc0c6a00 con 0x7f37dc1552c0 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37e0900640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f37dc15bce0 con 0x7f37dc0caec0 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37e0900640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f37dc0cda20 con 0x7f37dc0cc920 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37dad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc1552c0 0x7f37dc152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37dad9d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc1552c0 0x7f37dc152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42938/0 (socket says 172.21.15.44:42938) 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37dad9d640 1 -- 172.21.15.44:0/1991437073 learned_addr learned my addr 172.21.15.44:0/1991437073 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37da59c640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37dc0caec0 0x7f37dc1f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37d9d9b640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37dc0cc920 0x7f37dc152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37dad9d640 1 -- 172.21.15.44:0/1991437073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37dc0cc920 msgr2=0x7f37dc152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37dad9d640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37dc0cc920 0x7f37dc152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37dad9d640 1 -- 172.21.15.44:0/1991437073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37dc0caec0 msgr2=0x7f37dc1f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:19.985 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37dad9d640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37dc0caec0 0x7f37dc1f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37dad9d640 1 -- 172.21.15.44:0/1991437073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37dc151430 con 0x7f37dc1552c0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.981+0000 7f37dad9d640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc1552c0 0x7f37dc152ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f37d4001560 tx=0x7f37d400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.982+0000 7f37caffd640 1 -- 172.21.15.44:0/1991437073 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4003b70 con 0x7f37dc1552c0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.982+0000 7f37e0900640 1 -- 172.21.15.44:0/1991437073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37dc151710 con 0x7f37dc1552c0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.982+0000 7f37e0900640 1 -- 172.21.15.44:0/1991437073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37dc0cfd10 con 0x7f37dc1552c0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.982+0000 7f37caffd640 1 -- 172.21.15.44:0/1991437073 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d4003d30 con 0x7f37dc1552c0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.982+0000 7f37caffd640 1 -- 172.21.15.44:0/1991437073 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d400aa50 con 0x7f37dc1552c0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.983+0000 7f37e0900640 1 -- 172.21.15.44:0/1991437073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c0005290 con 0x7f37dc1552c0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.984+0000 7f37caffd640 1 -- 172.21.15.44:0/1991437073 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37d400abf0 con 0x7f37dc1552c0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.984+0000 7f37caffd640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f37bc06c920 0x7f37bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.985+0000 7f37caffd640 1 -- 172.21.15.44:0/1991437073 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f37d408bc60 con 0x7f37dc1552c0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.985+0000 7f37da59c640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f37bc06c920 0x7f37bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:19.986 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.985+0000 7f37da59c640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f37bc06c920 0x7f37bc06ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009a30 tx=0x7f37cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:19.987 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:19.989+0000 7f37caffd640 1 -- 172.21.15.44:0/1991437073 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f37d4059ed0 con 0x7f37dc1552c0 2024-04-22T07:06:20.174 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.176+0000 7f37e0900640 1 -- 172.21.15.44:0/1991437073 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f37c0005d90 con 0x7f37dc1552c0 2024-04-22T07:06:20.175 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.176+0000 7f37caffd640 1 -- 172.21.15.44:0/1991437073 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f37d40597d0 con 0x7f37dc1552c0 2024-04-22T07:06:20.175 INFO:teuthology.orchestra.run.smithi044.stdout:pg_num: 64 2024-04-22T07:06:20.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 -- 172.21.15.44:0/1991437073 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f37bc06c920 msgr2=0x7f37bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:20.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f37bc06c920 0x7f37bc06ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009a30 tx=0x7f37cc008040 comp rx=0 tx=0).stop 2024-04-22T07:06:20.178 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 -- 172.21.15.44:0/1991437073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc1552c0 msgr2=0x7f37dc152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:20.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc1552c0 0x7f37dc152ab0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f37d4001560 tx=0x7f37d400cc70 comp rx=0 tx=0).stop 2024-04-22T07:06:20.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 -- 172.21.15.44:0/1991437073 shutdown_connections 2024-04-22T07:06:20.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f37bc06c920 0x7f37bc06ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f37dc0cc920 0x7f37dc152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.179 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f37dc1552c0 0x7f37dc152ab0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 --2- 172.21.15.44:0/1991437073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37dc0caec0 0x7f37dc1f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 -- 172.21.15.44:0/1991437073 >> 172.21.15.44:0/1991437073 conn(0x7f37dc0676b0 msgr2=0x7f37dc0c5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:20.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 -- 172.21.15.44:0/1991437073 shutdown_connections 2024-04-22T07:06:20.182 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.179+0000 7f37e0900640 1 -- 172.21.15.44:0/1991437073 wait complete. 2024-04-22T07:06:20.193 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T07:06:20.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.298+0000 7f7aa2f3d640 1 Processor -- start 2024-04-22T07:06:20.297 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.298+0000 7f7aa2f3d640 1 -- start start 2024-04-22T07:06:20.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.299+0000 7f7aa2f3d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f7a9c0c32f0 0x7f7a9c0c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:20.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.299+0000 7f7aa2f3d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 0x7f7a9c0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:20.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.299+0000 7f7aa2f3d640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a9c0c9210 con 0x7f7a9c0ca4d0 2024-04-22T07:06:20.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.299+0000 7f7aa2f3d640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f7a9c06fee0 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.298 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.299+0000 7f7aa2f3d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9c0c94e0 con 0x7f7a9c0c5260 2024-04-22T07:06:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.300+0000 7f7aa14da640 1 --1- >> v1:172.21.15.44:6789/0 conn(0x7f7a9c0ca4d0 0x7f7a9c0c99e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.44:6789/0 says I am v1:172.21.15.44:32806/0 (socket says 172.21.15.44:32806) 2024-04-22T07:06:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.300+0000 7f7aa14da640 1 -- 172.21.15.44:0/3039197395 learned_addr learned my addr 172.21.15.44:0/3039197395 (peer_addr_for_me v1:172.21.15.44:0/0) 2024-04-22T07:06:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.300+0000 7f7aa0cd9640 1 --2- 172.21.15.44:0/3039197395 >> v2:172.21.15.44:3300/0 conn(0x7f7a9c0c32f0 0x7f7a9c0c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:20.299 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.300+0000 7f7a9bfff640 1 --2- 172.21.15.44:0/3039197395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 0x7f7a9c0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.300+0000 7f7a9b7fe640 1 -- 172.21.15.44:0/3039197395 <== mon.0 v1:172.21.15.44:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167522044 0 0) 0x7f7a9c0c9210 con 0x7f7a9c0ca4d0 2024-04-22T07:06:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.300+0000 7f7a9bfff640 1 -- 172.21.15.44:0/3039197395 >> v2:172.21.15.44:3300/0 conn(0x7f7a9c0c32f0 msgr2=0x7f7a9c0c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T07:06:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.300+0000 7f7a9bfff640 1 --2- 172.21.15.44:0/3039197395 >> v2:172.21.15.44:3300/0 conn(0x7f7a9c0c32f0 0x7f7a9c0c36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.300+0000 7f7a9bfff640 1 -- 172.21.15.44:0/3039197395 >> v1:172.21.15.44:6789/0 conn(0x7f7a9c0ca4d0 legacy=0x7f7a9c0c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.300+0000 7f7a9bfff640 1 -- 172.21.15.44:0/3039197395 --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9c0caa10 con 0x7f7a9c0c5260 2024-04-22T07:06:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.301+0000 7f7a9bfff640 1 --2- 172.21.15.44:0/3039197395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 0x7f7a9c0c8cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a880097a0 tx=0x7f7a8802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b620ca19709627 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.301+0000 7f7a9b7fe640 1 -- 172.21.15.44:0/3039197395 <== mon.2 v2:172.21.15.70:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88034070 con 0x7f7a9c0c5260 2024-04-22T07:06:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.301+0000 7f7a9b7fe640 1 -- 172.21.15.44:0/3039197395 <== mon.2 v2:172.21.15.70:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a88030800 con 0x7f7a9c0c5260 2024-04-22T07:06:20.300 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.301+0000 7f7a9b7fe640 1 -- 172.21.15.44:0/3039197395 <== mon.2 v2:172.21.15.70:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88030bd0 con 0x7f7a9c0c5260 2024-04-22T07:06:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.302+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3039197395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 msgr2=0x7f7a9c0c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.302+0000 7f7aa2f3d640 1 --2- 172.21.15.44:0/3039197395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 0x7f7a9c0c8cd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a880097a0 tx=0x7f7a8802b940 comp rx=0 tx=0).stop 2024-04-22T07:06:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.302+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3039197395 shutdown_connections 2024-04-22T07:06:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.302+0000 7f7aa2f3d640 1 --2- 172.21.15.44:0/3039197395 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 0x7f7a9c0c8cd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.302+0000 7f7aa2f3d640 1 --2- 172.21.15.44:0/3039197395 >> v2:172.21.15.44:3300/0 conn(0x7f7a9c0c32f0 0x7f7a9c0c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.301 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.302+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3039197395 >> 172.21.15.44:0/3039197395 conn(0x7f7a9c071850 msgr2=0x7f7a9c071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.302+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3039197395 shutdown_connections 2024-04-22T07:06:20.302 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.302+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3039197395 wait complete. 2024-04-22T07:06:20.303 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.303+0000 7f7aa2f3d640 1 Processor -- start 2024-04-22T07:06:20.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.303+0000 7f7aa2f3d640 1 -- start start 2024-04-22T07:06:20.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7aa2f3d640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a9c0c32f0 0x7f7a9c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:20.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7aa2f3d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 0x7f7a9c0cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:20.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7aa2f3d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9c0ca4d0 0x7f7a9c0cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:20.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7aa2f3d640 1 -- --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c0706f0 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7aa2f3d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c070410 con 0x7f7a9c0ca4d0 2024-04-22T07:06:20.304 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7aa2f3d640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9c153020 con 0x7f7a9c0c5260 2024-04-22T07:06:20.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7aa0cd9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a9c0c32f0 0x7f7a9c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:20.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7aa0cd9640 1 --2- >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a9c0c32f0 0x7f7a9c158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.44:3300/0 says I am v2:172.21.15.44:42950/0 (socket says 172.21.15.44:42950) 2024-04-22T07:06:20.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7aa0cd9640 1 -- 172.21.15.44:0/3283933792 learned_addr learned my addr 172.21.15.44:0/3283933792 (peer_addr_for_me v2:172.21.15.44:0/0) 2024-04-22T07:06:20.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7aa14da640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9c0ca4d0 0x7f7a9c0cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:20.305 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.304+0000 7f7a9bfff640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 0x7f7a9c0cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:20.315 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7aa0cd9640 1 -- 172.21.15.44:0/3283933792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 msgr2=0x7f7a9c0cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7aa0cd9640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 0x7f7a9c0cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7aa0cd9640 1 -- 172.21.15.44:0/3283933792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9c0ca4d0 msgr2=0x7f7a9c0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7aa0cd9640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9c0ca4d0 0x7f7a9c0cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7aa0cd9640 1 -- 172.21.15.44:0/3283933792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9c158df0 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7aa0cd9640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a9c0c32f0 0x7f7a9c158240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000e9b0 tx=0x7f7a9000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7a997fa640 1 -- 172.21.15.44:0/3283933792 <== mon.0 v2:172.21.15.44:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90017070 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7a997fa640 1 -- 172.21.15.44:0/3283933792 <== mon.0 v2:172.21.15.44:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9000c9a0 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7a997fa640 1 -- 172.21.15.44:0/3283933792 <== mon.0 v2:172.21.15.44:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000cd90 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.316 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3283933792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9c159090 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.305+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3283933792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9c159650 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.306+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3283933792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a9c15a240 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.308+0000 7f7a997fa640 1 -- 172.21.15.44:0/3283933792 <== mon.0 v2:172.21.15.44:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a9001a070 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.308+0000 7f7a997fa640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7a8006c920 0x7f7a8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:20.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.308+0000 7f7a997fa640 1 -- 172.21.15.44:0/3283933792 <== mon.0 v2:172.21.15.44:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5911+0+0 (secure 0 0 0) 0x7f7a9008bc00 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.308+0000 7f7a9bfff640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7a8006c920 0x7f7a8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T07:06:20.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.309+0000 7f7a9bfff640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7a8006c920 0x7f7a8006ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7a88036000 tx=0x7f7a8803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T07:06:20.317 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.314+0000 7f7a997fa640 1 -- 172.21.15.44:0/3283933792 <== mon.0 v2:172.21.15.44:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a9c15a240 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.562+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3283933792 --> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7a9c148a30 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.561 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.563+0000 7f7a997fa640 1 -- 172.21.15.44:0/3283933792 <== mon.0 v2:172.21.15.44:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v26) ==== 80+0+122 (secure 0 0 0) 0x7f7a90092080 con 0x7f7a9c0c32f0 2024-04-22T07:06:20.561 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.70:6834/1969018412 2024-04-23T07:05:49.227539+0000 2024-04-22T07:06:20.562 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.70:6832/1969018412 2024-04-23T07:05:49.227539+0000 2024-04-22T07:06:20.562 INFO:teuthology.orchestra.run.smithi044.stderr:listed 2 entries 2024-04-22T07:06:20.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3283933792 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7a8006c920 msgr2=0x7f7a8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:20.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7a8006c920 0x7f7a8006ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7a88036000 tx=0x7f7a8803b040 comp rx=0 tx=0).stop 2024-04-22T07:06:20.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3283933792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a9c0c32f0 msgr2=0x7f7a9c158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T07:06:20.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a9c0c32f0 0x7f7a9c158240 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000e9b0 tx=0x7f7a9000ee70 comp rx=0 tx=0).stop 2024-04-22T07:06:20.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3283933792 shutdown_connections 2024-04-22T07:06:20.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.44:6832/1889438723,v1:172.21.15.44:6833/1889438723] conn(0x7f7a8006c920 0x7f7a8006ede0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f7a9c0c5260 0x7f7a9c0cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a9c0ca4d0 0x7f7a9c0cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 --2- 172.21.15.44:0/3283933792 >> [v2:172.21.15.44:3300/0,v1:172.21.15.44:6789/0] conn(0x7f7a9c0c32f0 0x7f7a9c158240 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:20.564 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3283933792 >> 172.21.15.44:0/3283933792 conn(0x7f7a9c071850 msgr2=0x7f7a9c0c7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T07:06:20.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3283933792 shutdown_connections 2024-04-22T07:06:20.565 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:20.565+0000 7f7aa2f3d640 1 -- 172.21.15.44:0/3283933792 wait complete. 2024-04-22T07:06:20.578 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi044.front.sepia.ceph.com... 2024-04-22T07:06:20.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:06:20.579 DEBUG:teuthology.orchestra.run.smithi044:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-22T07:06:20.604 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:06:20.604 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:06:20.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:06:20.604 DEBUG:teuthology.orchestra.run.smithi044:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-22T07:06:22.179 INFO:tasks.ceph.osd.0:Sent signal 15 2024-04-22T07:06:22.179 INFO:tasks.ceph.osd.1:Sent signal 15 2024-04-22T07:06:22.179 INFO:tasks.ceph.osd.2:Sent signal 15 2024-04-22T07:06:22.179 INFO:tasks.ceph.osd.3:Sent signal 15 2024-04-22T07:06:22.180 INFO:tasks.ceph.osd.4:Sent signal 15 2024-04-22T07:06:22.180 INFO:tasks.ceph.osd.5:Sent signal 15 2024-04-22T07:06:22.180 INFO:tasks.ceph.osd.6:Sent signal 15 2024-04-22T07:06:22.180 INFO:tasks.ceph.osd.7:Sent signal 15 2024-04-22T07:06:22.181 INFO:tasks.ceph.mds.a:Sent signal 15 2024-04-22T07:06:22.181 INFO:tasks.ceph.mds.f:Sent signal 15 2024-04-22T07:06:22.181 INFO:tasks.ceph.mon.a:Sent signal 15 2024-04-22T07:06:22.181 INFO:tasks.ceph.mon.b:Sent signal 15 2024-04-22T07:06:22.181 INFO:tasks.ceph.mon.c:Sent signal 15 2024-04-22T07:06:22.182 INFO:tasks.ceph.mgr.y:Sent signal 15 2024-04-22T07:06:22.182 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-04-22T07:06:22.182 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T07:06:22.181+0000 7f82be241640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33450) UID: 0 2024-04-22T07:06:22.183 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T07:06:22.181+0000 7f82be241640 -1 osd.0 26 *** Got signal Terminated *** 2024-04-22T07:06:22.183 INFO:tasks.ceph.osd.0.smithi044.stderr:2024-04-22T07:06:22.181+0000 7f82be241640 -1 osd.0 26 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:06:22.183 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T07:06:22.181+0000 7fb5c1692640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33458) UID: 0 2024-04-22T07:06:22.183 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T07:06:22.181+0000 7fb5c1692640 -1 osd.1 26 *** Got signal Terminated *** 2024-04-22T07:06:22.183 INFO:tasks.ceph.osd.1.smithi044.stderr:2024-04-22T07:06:22.181+0000 7fb5c1692640 -1 osd.1 26 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:06:22.183 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T07:06:22.182+0000 7fecfa460640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33462) UID: 0 2024-04-22T07:06:22.183 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T07:06:22.182+0000 7fecfa460640 -1 osd.2 26 *** Got signal Terminated *** 2024-04-22T07:06:22.184 INFO:tasks.ceph.osd.2.smithi044.stderr:2024-04-22T07:06:22.182+0000 7fecfa460640 -1 osd.2 26 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:06:22.184 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T07:06:22.182+0000 7f12368d6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33471) UID: 0 2024-04-22T07:06:22.184 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T07:06:22.182+0000 7f12368d6640 -1 osd.3 26 *** Got signal Terminated *** 2024-04-22T07:06:22.184 INFO:tasks.ceph.osd.3.smithi044.stderr:2024-04-22T07:06:22.182+0000 7f12368d6640 -1 osd.3 26 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:06:22.184 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-04-22T07:06:22.183+0000 7f5234d70640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 39408) UID: 0 2024-04-22T07:06:22.184 INFO:tasks.ceph.mds.a.smithi044.stderr:2024-04-22T07:06:22.183+0000 7f5234d70640 -1 mds.a *** got signal Terminated *** 2024-04-22T07:06:22.185 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-04-22T07:06:22.183+0000 7f48b5dd9640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 33173) UID: 0 2024-04-22T07:06:22.185 INFO:tasks.ceph.mon.a.smithi044.stderr:2024-04-22T07:06:22.183+0000 7f48b5dd9640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-22T07:06:22.185 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:06:22.184+0000 7f74aad09640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i y (PID: 33209) UID: 0 2024-04-22T07:06:22.185 INFO:tasks.ceph.mgr.y.smithi044.stderr:2024-04-22T07:06:22.184+0000 7f74aad09640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-22T07:06:22.186 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T07:06:22.183+0000 7f6f3453f640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33644) UID: 0 2024-04-22T07:06:22.186 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T07:06:22.183+0000 7f6f3453f640 -1 osd.5 26 *** Got signal Terminated *** 2024-04-22T07:06:22.186 INFO:tasks.ceph.osd.5.smithi070.stderr:2024-04-22T07:06:22.183+0000 7f6f3453f640 -1 osd.5 26 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:06:22.186 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T07:06:22.183+0000 7fab931a3640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33640) UID: 0 2024-04-22T07:06:22.186 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T07:06:22.183+0000 7fab931a3640 -1 osd.4 26 *** Got signal Terminated *** 2024-04-22T07:06:22.186 INFO:tasks.ceph.osd.4.smithi070.stderr:2024-04-22T07:06:22.183+0000 7fab931a3640 -1 osd.4 26 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:06:22.187 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T07:06:22.184+0000 7fb9e2352640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33643) UID: 0 2024-04-22T07:06:22.187 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T07:06:22.184+0000 7fb9e2352640 -1 osd.6 26 *** Got signal Terminated *** 2024-04-22T07:06:22.187 INFO:tasks.ceph.osd.6.smithi070.stderr:2024-04-22T07:06:22.184+0000 7fb9e2352640 -1 osd.6 26 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:06:22.187 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T07:06:22.184+0000 7fcf5ed39640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33647) UID: 0 2024-04-22T07:06:22.187 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T07:06:22.184+0000 7fcf5ed39640 -1 osd.7 26 *** Got signal Terminated *** 2024-04-22T07:06:22.187 INFO:tasks.ceph.osd.7.smithi070.stderr:2024-04-22T07:06:22.184+0000 7fcf5ed39640 -1 osd.7 26 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T07:06:22.188 INFO:tasks.ceph.mds.f.smithi070.stderr:2024-04-22T07:06:22.184+0000 7f7903059640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i f (PID: 39101) UID: 0 2024-04-22T07:06:22.188 INFO:tasks.ceph.mds.f.smithi070.stderr:2024-04-22T07:06:22.184+0000 7f7903059640 -1 mds.f *** got signal Terminated *** 2024-04-22T07:06:22.188 INFO:tasks.ceph.mon.b.smithi070.stderr:2024-04-22T07:06:22.185+0000 7f4e66f7d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 33060) UID: 0 2024-04-22T07:06:22.188 INFO:tasks.ceph.mon.b.smithi070.stderr:2024-04-22T07:06:22.185+0000 7f4e66f7d640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-04-22T07:06:22.188 INFO:tasks.ceph.mon.c.smithi070.stderr:2024-04-22T07:06:22.185+0000 7ff797d4f640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 33062) UID: 0 2024-04-22T07:06:22.188 INFO:tasks.ceph.mon.c.smithi070.stderr:2024-04-22T07:06:22.185+0000 7ff797d4f640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-04-22T07:06:22.383 INFO:tasks.ceph.mgr.x.smithi070.stderr:daemon-helper: command crashed with signal 15 2024-04-22T07:06:23.495 INFO:tasks.workunit.client.0.smithi044.stderr:Updating files: 94% (11553/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-22T07:06:24.194 INFO:tasks.workunit.client.0.smithi044.stderr:Note: switching to '86f7587a5a09af35f0895e1a2d08527638fae697'. 2024-04-22T07:06:24.194 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T07:06:24.194 INFO:tasks.workunit.client.0.smithi044.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T07:06:24.194 INFO:tasks.workunit.client.0.smithi044.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T07:06:24.194 INFO:tasks.workunit.client.0.smithi044.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T07:06:24.194 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T07:06:24.194 INFO:tasks.workunit.client.0.smithi044.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T07:06:24.194 INFO:tasks.workunit.client.0.smithi044.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T07:06:24.195 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T07:06:24.195 INFO:tasks.workunit.client.0.smithi044.stderr: git switch -c 2024-04-22T07:06:24.195 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T07:06:24.195 INFO:tasks.workunit.client.0.smithi044.stderr:Or undo this operation with: 2024-04-22T07:06:24.195 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T07:06:24.195 INFO:tasks.workunit.client.0.smithi044.stderr: git switch - 2024-04-22T07:06:24.195 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T07:06:24.195 INFO:tasks.workunit.client.0.smithi044.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T07:06:24.195 INFO:tasks.workunit.client.0.smithi044.stderr: 2024-04-22T07:06:24.195 INFO:tasks.workunit.client.0.smithi044.stderr:HEAD is now at 86f7587a5a0 qa/fsx: use the master branch to build the xfstest-dev 2024-04-22T07:06:24.200 DEBUG:teuthology.orchestra.run.smithi044:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T07:06:24.224 INFO:tasks.workunit.client.0.smithi044.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T07:06:24.226 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T07:06:24.226 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T07:06:24.839 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T07:06:24.896 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T07:06:24.937 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T07:06:24.938 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T07:06:24.938 INFO:tasks.workunit.client.0.smithi044.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T07:06:24.978 INFO:tasks.workunit.client.0.smithi044.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T07:06:24.981 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:06:24.981 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T07:06:25.037 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2024-04-22T07:06:25.038 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2024-04-22T07:06:25.038 DEBUG:teuthology.orchestra.run.smithi044:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86f7587a5a09af35f0895e1a2d08527638fae697 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2024-04-22T07:06:25.095 INFO:tasks.workunit.client.0.smithi044.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2024-04-22T07:06:27.574 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir foo 2024-04-22T07:06:27.575 INFO:tasks.workunit.client.0.smithi044.stderr:+ cp linux-2.6.33.tar.bz2 foo 2024-04-22T07:06:27.614 INFO:tasks.workunit.client.0.smithi044.stderr:+ mkdir foo/.snap/barsnap 2024-04-22T07:06:27.615 INFO:tasks.workunit.client.0.smithi044.stderr:mkdir: cannot create directory ‘foo/.snap/barsnap’: No such file or directory 2024-04-22T07:06:27.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:06:27.616 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2024-04-22T07:06:27.616 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T07:06:27.909 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi044 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86f7587a5a09af35f0895e1a2d08527638fae697 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' 2024-04-22T07:06:28.136 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=fb091e6d9896408dba3fbece7b86b757 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi044 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86f7587a5a09af35f0895e1a2d08527638fae697 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh' 2024-04-22T07:06:28.139 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-22T07:06:28.148 INFO:tasks.quiescer:joining Quiescers 2024-04-22T07:06:28.149 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T07:06:28.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.266+0000 7fdf21231640 1 Processor -- start 2024-04-22T07:06:28.264 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.266+0000 7fdf21231640 1 -- start start 2024-04-22T07:06:28.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.267+0000 7fdf21231640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdf1c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.267+0000 7fdf21231640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdf1c0c8d60 2024-04-22T07:06:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.267+0000 7fdf21231640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdf1c15b360 2024-04-22T07:06:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.267+0000 7fdf21231640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdf1c155220 2024-04-22T07:06:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.267+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 msgr2=0x7fdf1c06b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.267+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdf1c06b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.267+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdf1c0c8d60 legacy=0x7fdf1c0ce310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:28.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.267+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdf1c15b360 legacy=0x7fdf1c15b790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:28.465 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.468+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdf1c0c8d60 legacy=0x7fdf1c0ce310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:28.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.468+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 msgr2=0x7fdf1c06b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:28.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.468+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdf1c06b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:28.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.468+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdf1c15b360 legacy=0x7fdf1c15b790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:28.866 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.869+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 msgr2=0x7fdf1c06b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:28.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.869+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdf1c0c8d60 legacy=0x7fdf1c0ce310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:28.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.869+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdf1c06b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:28.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:28.869+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdf1c15b360 legacy=0x7fdf1c15b790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:29.667 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:29.670+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 msgr2=0x7fdf1c06b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:29.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:29.670+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdf1c0c8d60 legacy=0x7fdf1c0ce310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:29.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:29.670+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdf1c06b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:29.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:29.670+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdf1c15b360 legacy=0x7fdf1c15b790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:31.265 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 msgr2=0x7fdf1c06b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:31.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdf1c06b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:31.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdf1c15b360 legacy=0x7fdf1c15b790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:31.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdf1c0c8d60 legacy=0x7fdf1c0ce310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:31.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 0x7fdefc005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:31.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 0x7fdefc0085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:31.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc006130 2024-04-22T07:06:31.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc0036a0 2024-04-22T07:06:31.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 msgr2=0x7fdefc0085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:31.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 msgr2=0x7fdefc005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:31.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 0x7fdefc0085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:31.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.267+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 0x7fdefc005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.468+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 msgr2=0x7fdefc005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.468+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 msgr2=0x7fdefc0085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.468+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 0x7fdefc005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:31.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.468+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 0x7fdefc0085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:31.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.869+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 msgr2=0x7fdefc0085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:31.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.869+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 msgr2=0x7fdefc005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:31.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.869+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 0x7fdefc0085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:31.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:31.869+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 0x7fdefc005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:32.670+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 msgr2=0x7fdefc005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:32.670+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 0x7fdefc005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:32.670+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 msgr2=0x7fdefc0085f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:32.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:32.670+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 0x7fdefc0085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 msgr2=0x7fdefc005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0036a0 0x7fdefc005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 msgr2=0x7fdefc0085f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc006130 0x7fdefc0085f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:34.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdefc00b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:06:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdf1c0c8d60 2024-04-22T07:06:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdf1c15b360 2024-04-22T07:06:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdf1c155220 2024-04-22T07:06:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 msgr2=0x7fdefc00b010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdefc00b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:34.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf1b59e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdf1c0c8d60 legacy=0x7fdefc010cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:34.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.267+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdf1c15b360 legacy=0x7fdefc00d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:34.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.468+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 msgr2=0x7fdefc00b010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:34.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.468+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdefc00b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:34.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.468+0000 7fdf1b59e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdf1c0c8d60 legacy=0x7fdefc010cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:34.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.468+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdf1c15b360 legacy=0x7fdefc00d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.869+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 msgr2=0x7fdefc00b010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.869+0000 7fdf1b59e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdf1c0c8d60 legacy=0x7fdefc010cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.869+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdefc00b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:34.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:34.869+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdf1c15b360 legacy=0x7fdefc00d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:35.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:35.670+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 msgr2=0x7fdefc00b010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:35.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:35.670+0000 7fdf1b59e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdf1c0c8d60 legacy=0x7fdefc010cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:35.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:35.670+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdefc00b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:35.668 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:35.670+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdf1c15b360 legacy=0x7fdefc00d580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:37.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 msgr2=0x7fdefc00b010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:37.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdf1c155220 0x7fdefc00b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:37.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdf1c15b360 legacy=0x7fdefc00d580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdf1c0c8d60 legacy=0x7fdefc010cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 0x7fdefc005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 0x7fdefc006b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc0066e0 2024-04-22T07:06:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc0086c0 2024-04-22T07:06:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 msgr2=0x7fdefc006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 msgr2=0x7fdefc005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:37.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 0x7fdefc006b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:37.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.268+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 0x7fdefc005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:37.466 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.469+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 msgr2=0x7fdefc006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.469+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 msgr2=0x7fdefc005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.469+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 0x7fdefc006b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:37.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.469+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 0x7fdefc005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:37.867 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.870+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 msgr2=0x7fdefc006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.870+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 msgr2=0x7fdefc005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.870+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 0x7fdefc006b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:37.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:37.870+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 0x7fdefc005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:38.671+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 msgr2=0x7fdefc005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:38.671+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 msgr2=0x7fdefc006b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:38.671+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 0x7fdefc005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:38.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:38.671+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 0x7fdefc006b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:40.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 msgr2=0x7fdefc005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc0086c0 0x7fdefc005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 msgr2=0x7fdefc006b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0066e0 0x7fdefc006b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:06:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 0x7fdefc00afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:40.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc006b50 2024-04-22T07:06:40.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc007f30 2024-04-22T07:06:40.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc007610 2024-04-22T07:06:40.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 msgr2=0x7fdefc00afd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:40.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 0x7fdefc00afd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:40.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 msgr2=0x7fdefc0079f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:40.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:40.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.268+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.469+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 msgr2=0x7fdefc00afd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.469+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 0x7fdefc00afd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.469+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 msgr2=0x7fdefc0079f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.469+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:40.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.469+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:40.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.870+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 msgr2=0x7fdefc00afd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:40.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.870+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 0x7fdefc00afd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:40.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.870+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 msgr2=0x7fdefc0079f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:40.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.870+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:40.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:40.870+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:41.170 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:06:41.174 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:06:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:41.671+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 msgr2=0x7fdefc00afd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:41.671+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 0x7fdefc00afd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:41.671+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 msgr2=0x7fdefc0079f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:41.671+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:41.669 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:41.671+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:43.266 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 msgr2=0x7fdefc0079f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:43.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:43.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 msgr2=0x7fdefc00afd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc007f30 0x7fdefc00afd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 0x7fdefc005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc005b50 2024-04-22T07:06:43.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc005200 2024-04-22T07:06:43.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc005b50 legacy=0x7fdefc00f590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:43.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:43.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.269+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 0x7fdefc005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:43.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.470+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc005b50 legacy=0x7fdefc00f590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:43.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.470+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:43.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.470+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 0x7fdefc005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:43.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.871+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc005b50 legacy=0x7fdefc00f590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:43.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.871+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:43.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:43.871+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 0x7fdefc005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:44.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:44.672+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc005b50 legacy=0x7fdefc00f590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:44.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:44.672+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc005610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:44.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:44.672+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 0x7fdefc005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:46.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc005610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005200 0x7fdefc005610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc005b50 legacy=0x7fdefc00f590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:46.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:06:46.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:46.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc006b50 2024-04-22T07:06:46.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc005200 2024-04-22T07:06:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc007610 2024-04-22T07:06:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc00afb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:46.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:46.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 msgr2=0x7fdefc0079f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:46.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.269+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.470+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc00afb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:46.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.470+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:46.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.470+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:46.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.470+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 msgr2=0x7fdefc0079f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:46.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.470+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:46.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.871+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc00afb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:46.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.871+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:46.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.871+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:46.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.871+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 msgr2=0x7fdefc0079f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:46.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:46.871+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:47.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:47.672+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc00afb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:47.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:47.672+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:47.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:47.672+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 msgr2=0x7fdefc0079f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:47.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:47.672+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:47.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:47.672+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:49.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 msgr2=0x7fdefc0079f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:49.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007610 0x7fdefc0079f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:49.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc00afb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:49.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:49.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:49.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 0x7fdefc010c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:49.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc0055e0 2024-04-22T07:06:49.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc005a50 2024-04-22T07:06:49.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0055e0 legacy=0x7fdefc00d520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:49.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 msgr2=0x7fdefc010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:49.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.269+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 0x7fdefc010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:49.467 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.470+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0055e0 legacy=0x7fdefc00d520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:49.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.470+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 msgr2=0x7fdefc010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:49.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.470+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 0x7fdefc010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:49.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.871+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0055e0 legacy=0x7fdefc00d520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.871+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 msgr2=0x7fdefc010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:49.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:49.871+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 0x7fdefc010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:50.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:50.672+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0055e0 legacy=0x7fdefc00d520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:50.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:50.672+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 msgr2=0x7fdefc010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:50.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:50.672+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 0x7fdefc010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:52.267 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 msgr2=0x7fdefc010c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:52.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a50 0x7fdefc010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:52.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0055e0 legacy=0x7fdefc00d520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:52.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:52.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:06:52.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc011190 2024-04-22T07:06:52.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc006b50 2024-04-22T07:06:52.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc005200 2024-04-22T07:06:52.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc00afb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:52.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:52.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc011190 legacy=0x7fdefc011570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:52.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.269+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:52.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.470+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc00afb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:52.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.470+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:52.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.470+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc011190 legacy=0x7fdefc011570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:52.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.470+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:52.868 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.871+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc00afb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:52.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.871+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:52.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.871+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc011190 legacy=0x7fdefc011570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:52.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:52.871+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:53.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:53.672+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc00afb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:06:53.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:53.672+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:53.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:53.672+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc011190 legacy=0x7fdefc011570 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:06:53.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:53.672+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 msgr2=0x7fdefc00afb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc005200 0x7fdefc00afb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc011190 legacy=0x7fdefc011570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 0x7fdefc007610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 0x7fdefc00d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:55.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc005a20 2024-04-22T07:06:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc007b50 2024-04-22T07:06:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 msgr2=0x7fdefc00d500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:55.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.270+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 0x7fdefc00d500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:55.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.471+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:55.468 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.471+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 msgr2=0x7fdefc00d500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.471+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:55.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.471+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 0x7fdefc00d500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:55.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.872+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 msgr2=0x7fdefc00d500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:55.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.872+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:55.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.872+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 0x7fdefc00d500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:55.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:55.872+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:56.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:56.673+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:56.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:56.673+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:56.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:56.673+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 msgr2=0x7fdefc00d500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:56.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:56.673+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 0x7fdefc00d500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:58.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 msgr2=0x7fdefc00d500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:58.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc007b50 0x7fdefc00d500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:58.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:06:58.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc005a20 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:06:58.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 0x7fdefc00fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:58.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:06:58.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:06:58.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc006b50 2024-04-22T07:06:58.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc011c80 2024-04-22T07:06:58.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc004af0 2024-04-22T07:06:58.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:58.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 msgr2=0x7fdefc00fa50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:58.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:58.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 0x7fdefc00fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:58.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.270+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:06:58.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.471+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 msgr2=0x7fdefc00fa50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:58.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.471+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:58.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.471+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:58.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.471+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 0x7fdefc00fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:58.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.471+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:06:58.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.872+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:58.869 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.872+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 msgr2=0x7fdefc00fa50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:58.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.872+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:58.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.872+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:58.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:58.872+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 0x7fdefc00fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:06:59.670 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:59.673+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:06:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:59.673+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:06:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:59.673+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 msgr2=0x7fdefc00fa50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:06:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:59.673+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 0x7fdefc00fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:06:59.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:06:59.673+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:01.268 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 msgr2=0x7fdefc00fa50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011c80 0x7fdefc00fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc006b50 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc00cf80 2024-04-22T07:07:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc0116b0 2024-04-22T07:07:01.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc00ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:01.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:01.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.271+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:01.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.472+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc00ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.472+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:01.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.472+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.873+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.873+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc00ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:01.873+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:02.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:02.674+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc00ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:02.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:02.674+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:02.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:02.674+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:04.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.270+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:04.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.270+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.270+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc00ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.270+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:07:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.270+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.271+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.271+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc00cf80 2024-04-22T07:07:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.271+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc0116b0 2024-04-22T07:07:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.271+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc004af0 2024-04-22T07:07:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.271+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc00ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:04.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.271+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:04.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.271+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:04.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.271+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:04.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.271+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:04.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.472+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc00ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:04.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.472+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:04.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.472+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:04.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.472+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:04.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.472+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:04.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.873+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc00ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.873+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.873+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.873+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:04.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:04.873+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:05.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:05.674+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc00ff90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:05.674+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:05.674+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:05.674+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:05.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:05.674+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:07.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc0116b0 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc00ff90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 0x7fdefc011ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc00f9e0 2024-04-22T07:07:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc006b50 2024-04-22T07:07:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 msgr2=0x7fdefc011ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 0x7fdefc011ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:07.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.271+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc00f9e0 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:07.469 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.472+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 msgr2=0x7fdefc011ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:07.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.472+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 0x7fdefc011ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:07.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.472+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc00f9e0 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:07.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.873+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 msgr2=0x7fdefc011ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:07.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.873+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 0x7fdefc011ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:07.870 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:07.873+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc00f9e0 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:08.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:08.674+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 msgr2=0x7fdefc011ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:08.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:08.674+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 0x7fdefc011ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:08.671 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:08.674+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc00f9e0 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:10.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 msgr2=0x7fdefc011ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:10.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc006b50 0x7fdefc011ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:10.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc00f9e0 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:10.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc01c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:10.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:07:10.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:10.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc016480 2024-04-22T07:07:10.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc019f90 2024-04-22T07:07:10.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc004af0 2024-04-22T07:07:10.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 msgr2=0x7fdefc01c380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:10.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc01c380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:10.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:10.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc016480 legacy=0x7fdefc018870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:10.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.272+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.473+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 msgr2=0x7fdefc01c380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.473+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc01c380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.473+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.473+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc016480 legacy=0x7fdefc018870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:10.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.473+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:10.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.874+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 msgr2=0x7fdefc01c380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:10.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.874+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc01c380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:10.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.874+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:10.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.874+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc016480 legacy=0x7fdefc018870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:10.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:10.874+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:11.306 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:07:11.309 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:07:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:11.674+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 msgr2=0x7fdefc01c380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:11.672 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:11.674+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc01c380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:11.674+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc016480 legacy=0x7fdefc018870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:11.674+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:11.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:11.674+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:13.269 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 msgr2=0x7fdefc004ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004af0 0x7fdefc004ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 msgr2=0x7fdefc01c380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc01c380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc016480 legacy=0x7fdefc018870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc00d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:13.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc011ac0 2024-04-22T07:07:13.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00cf80 2024-04-22T07:07:13.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc011ac0 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:13.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:13.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.271+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:13.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.472+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc011ac0 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:13.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.472+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:13.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.472+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:13.871 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.873+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc011ac0 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:13.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.873+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:13.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:13.873+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:14.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:14.674+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc011ac0 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:14.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:14.674+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:14.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:14.674+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:16.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.270+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc00d3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:16.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc00d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:16.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc011ac0 legacy=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc00ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:07:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc007b50 2024-04-22T07:07:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc019f90 2024-04-22T07:07:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc00d3c0 2024-04-22T07:07:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 msgr2=0x7fdefc00ff10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc00ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:16.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:16.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.271+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:16.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.472+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 msgr2=0x7fdefc00ff10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:16.470 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.472+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc00ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:16.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.472+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:16.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.472+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:16.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.472+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.873+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 msgr2=0x7fdefc00ff10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.873+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc00ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.873+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.873+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:16.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:16.873+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:17.674+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 msgr2=0x7fdefc00ff10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:17.674+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc00ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:17.674+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:17.674+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:17.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:17.674+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:19.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:19.270 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:19.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 msgr2=0x7fdefc00ff10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:19.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc019f90 0x7fdefc00ff10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:19.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:19.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 0x7fdefc010860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:19.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc00cf80 2024-04-22T07:07:19.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc010450 2024-04-22T07:07:19.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:19.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 msgr2=0x7fdefc010860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:19.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.272+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 0x7fdefc010860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:19.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.473+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.473+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 msgr2=0x7fdefc010860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:19.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.473+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 0x7fdefc010860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.874+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.874+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 msgr2=0x7fdefc010860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:19.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:19.874+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 0x7fdefc010860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:20.675+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:20.675+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 msgr2=0x7fdefc010860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:20.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:20.675+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 0x7fdefc010860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:22.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 msgr2=0x7fdefc010860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:22.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010450 0x7fdefc010860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:22.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:22.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:22.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:07:22.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:22.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc007b50 2024-04-22T07:07:22.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00ff10 2024-04-22T07:07:22.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc00d3c0 2024-04-22T07:07:22.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:22.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:22.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:22.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:22.314 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.272+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:22.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.473+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:22.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.473+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:22.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.473+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:22.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.473+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:22.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.473+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.874+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.874+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.874+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.874+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:22.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:22.874+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:23.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:23.675+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:23.675+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:23.675+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:23.675+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:23.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:23.675+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:25.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:25.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:25.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:25.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:25.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:25.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:25.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc010d70 2024-04-22T07:07:25.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00cf80 2024-04-22T07:07:25.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010d70 legacy=0x7fdefc011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:25.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:25.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.272+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.473+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010d70 legacy=0x7fdefc011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.473+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:25.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.473+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:25.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.874+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010d70 legacy=0x7fdefc011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:25.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.874+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:25.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:25.874+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:26.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:26.675+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010d70 legacy=0x7fdefc011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:26.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:26.675+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:26.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:26.675+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:28.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:28.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:28.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010d70 legacy=0x7fdefc011180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:28.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:28.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:07:28.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc010d70 2024-04-22T07:07:28.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc007b50 2024-04-22T07:07:28.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00ff10 2024-04-22T07:07:28.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:28.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:28.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010d70 legacy=0x7fdefc011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:28.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.273+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:28.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.474+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010d70 legacy=0x7fdefc011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.474+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.474+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:28.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.474+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:28.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.875+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010d70 legacy=0x7fdefc011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:28.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.875+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:28.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.875+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:28.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:28.875+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:29.673 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:29.676+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010d70 legacy=0x7fdefc011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:29.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:29.676+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:29.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:29.676+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:29.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:29.676+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:31.271 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:31.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:31.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:31.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010d70 legacy=0x7fdefc011180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:31.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:31.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:31.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00cf80 2024-04-22T07:07:31.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc00d3c0 2024-04-22T07:07:31.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:31.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:31.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:31.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.273+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:31.471 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.474+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.474+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.474+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:31.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.474+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:31.872 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.875+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:31.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.875+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:31.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.875+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:31.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:31.875+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:32.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:32.676+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:32.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:32.676+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:32.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:32.676+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:32.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:32.676+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:34.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:34.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:34.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 msgr2=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:34.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00cf80 0x7fdefc010830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:34.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:34.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc007b50 2024-04-22T07:07:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00ff10 2024-04-22T07:07:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc00d3c0 2024-04-22T07:07:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:07:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:34.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:34.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.273+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:34.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.474+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:34.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.474+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:34.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.474+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:34.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.474+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:34.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.474+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.875+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.875+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.875+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.875+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:34.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:34.875+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:35.676+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:35.676+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:35.676+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:35.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:35.676+0000 7fdf1b59e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:35.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:35.676+0000 7fdf1b59e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.273+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.273+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.273+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.273+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:37.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.273+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.273+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.273+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc00cf80 2024-04-22T07:07:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.273+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc011180 2024-04-22T07:07:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.274+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.274+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:37.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.274+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:37.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.475+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:37.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.475+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:37.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.475+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:37.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.876+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:37.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.876+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:37.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:37.876+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:38.674 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:38.677+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:38.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:38.677+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:38.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:38.677+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:40.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:40.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:40.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:40.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:40.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:07:40.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:40.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc007b50 2024-04-22T07:07:40.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00ff10 2024-04-22T07:07:40.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc011180 2024-04-22T07:07:40.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:40.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:40.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:40.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:40.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.274+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:40.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.475+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:40.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.475+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:40.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.475+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:40.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.475+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:40.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.475+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:40.873 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.876+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:40.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.876+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:40.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.876+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:40.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.876+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:40.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:40.876+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:41.425 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:07:41.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:07:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:41.677+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:41.677+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:41.677+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:41.677+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:41.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:41.677+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:43.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:43.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:43.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:43.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:43.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:43.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:43.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc00cf80 2024-04-22T07:07:43.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00d3c0 2024-04-22T07:07:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:43.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.274+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:43.472 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.475+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:43.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.475+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:43.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.475+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:43.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.876+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:43.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.876+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:43.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:43.876+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:44.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:44.677+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:44.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:44.677+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:44.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:44.677+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:46.272 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:46.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:46.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00cf80 legacy=0x7fdefc010830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:46.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:46.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc017490 2024-04-22T07:07:46.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc007b50 2024-04-22T07:07:46.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00ff10 2024-04-22T07:07:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:07:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc017490 legacy=0x7fdefc019880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:46.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.275+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:46.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.476+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc017490 legacy=0x7fdefc019880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:46.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.476+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:46.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.476+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:46.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.476+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:46.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.877+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc017490 legacy=0x7fdefc019880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:46.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.877+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:46.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.877+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:46.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:46.877+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:47.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:47.678+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc017490 legacy=0x7fdefc019880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:47.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:47.678+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:47.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:47.678+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:47.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:47.678+0000 7fdf1b59e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:49.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:49.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:49.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007b50 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:49.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc017490 legacy=0x7fdefc019880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:49.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:49.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:49.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00d3c0 2024-04-22T07:07:49.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc011180 2024-04-22T07:07:49.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:49.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:49.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:49.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.275+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:49.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.476+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:49.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.476+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:49.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.476+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:49.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.476+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.877+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:49.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.877+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.877+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:49.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:49.877+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:50.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:50.678+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:50.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:50.678+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:50.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:50.678+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:50.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:50.678+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:52.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:52.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:52.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 msgr2=0x7fdefc007610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:52.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc00d3c0 0x7fdefc007610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 0x7fdefc010c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00ff10 2024-04-22T07:07:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc010830 2024-04-22T07:07:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc0051e0 con 0x7fdefc011180 2024-04-22T07:07:52.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:07:52.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:52.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:52.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 msgr2=0x7fdefc010c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:52.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:52.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 0x7fdefc010c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:52.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.275+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:52.473 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.476+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.476+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.476+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 msgr2=0x7fdefc010c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.476+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.476+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 0x7fdefc010c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:52.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.476+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:52.874 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.877+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.877+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.877+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 msgr2=0x7fdefc010c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.877+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.877+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 0x7fdefc010c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:52.877+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:53.675 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:53.678+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:07:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:53.678+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:53.678+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 msgr2=0x7fdefc010c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:53.678+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:07:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:53.678+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 0x7fdefc010c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:53.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:53.678+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:07:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.276+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:55.273 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.276+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.276+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 msgr2=0x7fdefc010c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.276+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc010830 0x7fdefc010c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.276+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 msgr2=0x7fdefc0102f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.276+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc00ff10 0x7fdefc0102f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:07:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.276+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc0079f0 2024-04-22T07:07:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.276+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc007610 2024-04-22T07:07:55.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.276+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc016480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:55.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.276+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007610 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:55.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.477+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc016480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:55.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.477+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007610 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:55.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.878+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc016480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:55.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:55.878+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007610 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:56.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:56.679+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc016480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:56.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:56.679+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007610 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:58.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc007610 legacy=0x7fdefc003ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:58.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc016480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:07:58.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:07:58.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:07:58.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc0079f0 2024-04-22T07:07:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc017ba0 2024-04-22T07:07:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc011180 2024-04-22T07:07:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc016480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc017ba0 legacy=0x7fdefc019f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:58.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:58.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.276+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:07:58.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.477+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc016480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:58.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.477+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc017ba0 legacy=0x7fdefc019f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:58.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.477+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:58.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.477+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:07:58.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.878+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc016480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:58.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.878+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc017ba0 legacy=0x7fdefc019f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:58.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.878+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:58.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:58.878+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:07:59.676 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:59.679+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc016480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:07:59.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:59.679+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc017ba0 legacy=0x7fdefc019f90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:07:59.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:59.679+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:07:59.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:07:59.679+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:01.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc017ba0 legacy=0x7fdefc019f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc016480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 0x7fdefc010370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 0x7fdefc003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc0108b0 2024-04-22T07:08:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc00ff10 2024-04-22T07:08:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:01.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 msgr2=0x7fdefc010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:01.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.276+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 0x7fdefc010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:01.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.477+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:01.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.477+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:01.474 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.477+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 msgr2=0x7fdefc010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:01.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.477+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 0x7fdefc010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:01.875 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.878+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:01.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.878+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:01.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.878+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 msgr2=0x7fdefc010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:01.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:01.878+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 0x7fdefc010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:02.679+0000 7fdf1a59c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:02.679+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:02.679+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 msgr2=0x7fdefc010370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:02.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:02.679+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 0x7fdefc010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:04.274 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 msgr2=0x7fdefc010370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:04.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00ff10 0x7fdefc010370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:04.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:04.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc0108b0 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:04.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:04.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 0x7fdefc003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:04.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:04.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc010c90 2024-04-22T07:08:04.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:08:04.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc011180 2024-04-22T07:08:04.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc0051e0 con 0x7fdefc004520 2024-04-22T07:08:04.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:04.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:04.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:04.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:04.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:04.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.277+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:04.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.478+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:04.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.478+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:04.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.478+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:04.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.478+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:04.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.478+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:04.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.478+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:04.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.879+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:04.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.879+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:04.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.879+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:04.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.879+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:04.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.879+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:04.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:04.879+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:05.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:05.680+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:05.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:05.680+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:05.680+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:05.680+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:05.680+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:05.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:05.680+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.277+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.277+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc004520 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.277+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:07.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.277+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.277+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc00ff10 2024-04-22T07:08:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.277+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c15ae60 con 0x7fdefc0079f0 2024-04-22T07:08:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.277+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00ff10 legacy=0x7fdefc004900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:07.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.277+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:07.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.478+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00ff10 legacy=0x7fdefc004900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:07.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.478+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:07.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.879+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00ff10 legacy=0x7fdefc004900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:07.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:07.879+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:08.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:08.680+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00ff10 legacy=0x7fdefc004900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:08.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:08.680+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc00ff10 legacy=0x7fdefc004900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:08:10.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc0079f0 2024-04-22T07:08:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc010c90 2024-04-22T07:08:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc011180 2024-04-22T07:08:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:10.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.277+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.478+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:10.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.478+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.478+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.478+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:10.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.478+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:10.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.879+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:10.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.879+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:10.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.879+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:10.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.879+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:10.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:10.879+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:11.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:08:11.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:08:11.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:11.680+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:11.680+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:11.680+0000 7fdf1b59e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:11.680+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:11.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:11.680+0000 7fdf1a59c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:13.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:13.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:13.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:13.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:13.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 0x7fdefc003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc010370 2024-04-22T07:08:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00fed0 2024-04-22T07:08:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:13.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.276+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:13.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.477+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:13.475 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.477+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:13.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.477+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:13.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.878+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:13.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.878+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:13.876 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:13.878+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:14.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:14.679+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:14.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:14.679+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:14.677 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:14.679+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:16.275 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fed0 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:08:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc0079f0 2024-04-22T07:08:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc010c90 2024-04-22T07:08:16.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc011180 2024-04-22T07:08:16.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:16.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:16.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:16.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:16.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.277+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:16.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.478+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:16.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.478+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:16.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.478+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:16.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.478+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:16.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.478+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:16.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.879+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:16.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.879+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.879+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.879+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:16.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:16.879+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:17.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:17.680+0000 7fdf1b59e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:17.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:17.680+0000 7fdf1b59e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:17.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:17.680+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:17.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:17.680+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:17.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:17.680+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf18d99640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc011560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc011560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf18d99640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:19.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:19.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc010370 2024-04-22T07:08:19.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc00fe30 2024-04-22T07:08:19.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:19.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:19.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.277+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.478+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.478+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:19.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.478+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:19.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.879+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.879+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:19.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:19.879+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:20.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:20.680+0000 7fdf1a59c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:20.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:20.680+0000 7fdf1ad9d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:20.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:20.680+0000 7fdf1ad9d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:22.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 msgr2=0x7fdefc003aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:22.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:22.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:22.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:22.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf1b59e640 1 -- reap_dead start 2024-04-22T07:08:22.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:22.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf18d99640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c155600 con 0x7fdefc010370 2024-04-22T07:08:22.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf18d99640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc010c90 2024-04-22T07:08:22.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf18d99640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc008ca0 con 0x7fdefc00fe30 2024-04-22T07:08:22.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:22.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:22.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf1b59e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:22.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 msgr2=0x7fdefc003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:22.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.277+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:22.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.478+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:22.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.478+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:22.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.478+0000 7fdf1b59e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:22.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.478+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 msgr2=0x7fdefc003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:22.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.478+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.879+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.879+0000 7fdf1b59e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.879+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.879+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 msgr2=0x7fdefc003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:22.877 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:22.879+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:23.680+0000 7fdf1ad9d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:23.680+0000 7fdf1b59e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:23.680+0000 7fdf1ad9d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:23.680+0000 7fdf1a59c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 msgr2=0x7fdefc003a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:23.678 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:23.680+0000 7fdf1a59c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:25.276 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf18d99640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 msgr2=0x7fdefc003a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:25.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf18d99640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7fdefc00fe30 0x7fdefc003a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:25.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf18d99640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 msgr2=0x7fdefc003fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:25.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf18d99640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7fdefc010c90 0x7fdefc003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:25.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf18d99640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7fdefc010370 legacy=0x7fdefc00ef00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:25.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf18d99640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc004520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:25.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf18d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf1c0ca1c0 con 0x7fdefc0079f0 2024-04-22T07:08:25.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf18d99640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7fdf1c0c9ee0 con 0x7fdefc011180 2024-04-22T07:08:25.277 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc004520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:25.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:25.278 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.278+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc004520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:25.476 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.479+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc004520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.479+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:25.477 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.479+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc004520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:25.878 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.880+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.880+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc004520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:25.879 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:25.880+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc004520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:26.679 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:26.682+0000 7fdf1ad9d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7fdefc0079f0 legacy=0x7fdefc00e7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:26.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:26.682+0000 7fdf1a59c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 msgr2=0x7fdefc004520 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:26.680 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:26.682+0000 7fdf1a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdefc011180 0x7fdefc004520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:28.193 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-22T07:08:28.194 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-22T07:08:28.205 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-22T07:08:28.215 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-22T07:08:28.215 DEBUG:teuthology.orchestra.run.smithi044:> set -ex 2024-04-22T07:08:28.215 DEBUG:teuthology.orchestra.run.smithi044:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T07:08:28.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19631 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-04-22T07:08:28.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:08:28.248 DEBUG:teuthology.orchestra.run.smithi044:> ip netns list 2024-04-22T07:08:28.306 INFO:teuthology.orchestra.run.smithi044.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-22T07:08:28.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:08:28.307 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T07:08:28.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T07:08:28.376 DEBUG:teuthology.orchestra.run.smithi044:> sudo ip link delete ceph-brx 2024-04-22T07:08:28.476 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-22T07:08:28.485 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-22T07:08:28.486 DEBUG:teuthology.orchestra.run.smithi044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T07:08:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.596+0000 7f2d40481640 1 Processor -- start 2024-04-22T07:08:28.594 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.596+0000 7f2d40481640 1 -- start start 2024-04-22T07:08:28.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.597+0000 7f2d40481640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d38158240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:28.595 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.597+0000 7f2d40481640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 0x7f2d3806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.597+0000 7f2d40481640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d38151e90 2024-04-22T07:08:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.597+0000 7f2d40481640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d38072060 2024-04-22T07:08:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.597+0000 7f2d40481640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d38151ab0 2024-04-22T07:08:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.597+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d38151e90 legacy=0x7f2d380c3670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.597+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 msgr2=0x7f2d3806b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.597+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 0x7f2d3806b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.597+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 msgr2=0x7f2d38158240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:28.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.598+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d38158240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:28.795 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.798+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d38151e90 legacy=0x7f2d380c3670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:28.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.798+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 msgr2=0x7f2d3806b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:28.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.798+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 0x7f2d3806b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:28.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.799+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 msgr2=0x7f2d38158240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:28.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:28.799+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d38158240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:29.196 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:29.199+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d38151e90 legacy=0x7f2d380c3670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:29.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:29.199+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 msgr2=0x7f2d3806b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:29.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:29.199+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 0x7f2d3806b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:29.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:29.200+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 msgr2=0x7f2d38158240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:29.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:29.200+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d38158240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:29.997 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:30.000+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 msgr2=0x7f2d3806b970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:29.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:30.000+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d38151e90 legacy=0x7f2d380c3670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:29.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:30.000+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 0x7f2d3806b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:29.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:30.001+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 msgr2=0x7f2d38158240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:29.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:30.001+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d38158240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.597+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 msgr2=0x7f2d38158240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.597+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d38158240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.597+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 msgr2=0x7f2d3806b970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.597+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d38072060 0x7f2d3806b970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.597+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d38151e90 legacy=0x7f2d380c3670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:31.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.597+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 0x7f2d24009790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:31.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.597+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380c5690 con 0x7f2d240036b0 2024-04-22T07:08:31.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.597+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d240072d0 2024-04-22T07:08:31.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.598+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 msgr2=0x7f2d24009790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:31.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.598+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240036b0 legacy=0x7f2d24005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:31.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.598+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 0x7f2d24009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:31.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.799+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240036b0 legacy=0x7f2d24005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:31.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.799+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 msgr2=0x7f2d24009790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:31.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:31.799+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 0x7f2d24009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:32.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:32.200+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240036b0 legacy=0x7f2d24005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:32.200+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 msgr2=0x7f2d24009790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:32.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:32.200+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 0x7f2d24009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:33.001+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240036b0 legacy=0x7f2d24005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:33.001+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 msgr2=0x7f2d24009790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:32.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:33.001+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 0x7f2d24009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 msgr2=0x7f2d24009790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d240072d0 0x7f2d24009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:34.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240036b0 legacy=0x7f2d24005b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d24009ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:08:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d2400a8b0 2024-04-22T07:08:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d38151ab0 2024-04-22T07:08:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d2400a3e0 2024-04-22T07:08:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d2400a8b0 legacy=0x7f2d24005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 msgr2=0x7f2d24009790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:34.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 msgr2=0x7f2d24009ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:34.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:34.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.598+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d24009ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:34.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.799+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d2400a8b0 legacy=0x7f2d24005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.799+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 msgr2=0x7f2d24009790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.799+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 msgr2=0x7f2d24009ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.799+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:34.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:34.799+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d24009ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:35.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:35.200+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d2400a8b0 legacy=0x7f2d24005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:35.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:35.200+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 msgr2=0x7f2d24009790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:35.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:35.200+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 msgr2=0x7f2d24009ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:35.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:35.200+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:35.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:35.200+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d24009ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:35.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:36.001+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d2400a8b0 legacy=0x7f2d24005b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:35.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:36.001+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 msgr2=0x7f2d24009790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:35.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:36.001+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 msgr2=0x7f2d24009ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:35.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:36.001+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:35.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:36.001+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d24009ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:37.596 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 msgr2=0x7f2d24009790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:37.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:37.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 msgr2=0x7f2d24009ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:37.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d38151ab0 0x7f2d24009ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:37.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d2400a8b0 legacy=0x7f2d24005b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:37.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:37.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380c5690 con 0x7f2d2400d180 2024-04-22T07:08:37.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d240112d0 2024-04-22T07:08:37.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:37.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:37.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.598+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d180 legacy=0x7f2d2400d590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:37.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.799+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:37.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.799+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:37.796 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:37.799+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d180 legacy=0x7f2d2400d590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:38.197 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:38.200+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:38.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:38.200+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:38.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:38.200+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d180 legacy=0x7f2d2400d590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:38.998 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:39.001+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:38.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:39.001+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:38.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:39.001+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d180 legacy=0x7f2d2400d590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:40.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.598+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:40.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.598+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:40.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.598+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d180 legacy=0x7f2d2400d590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:40.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.598+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:08:40.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.598+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:40.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.598+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:40.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.598+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d24009ea0 2024-04-22T07:08:40.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.598+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:40.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.598+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d240112d0 2024-04-22T07:08:40.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.598+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d2400a3e0 2024-04-22T07:08:40.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.599+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:40.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.599+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 msgr2=0x7f2d24009770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:40.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.599+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:40.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.599+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:40.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.800+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.800+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.800+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 msgr2=0x7f2d24009770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.800+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:40.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:40.800+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:41.201+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:41.201+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:41.201+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 msgr2=0x7f2d24009770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:41.201+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:41.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:41.201+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:41.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:08:41.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:08:41.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:42.002+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:42.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:42.002+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:42.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:42.002+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 msgr2=0x7f2d24009770 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:42.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:42.002+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:42.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:42.002+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 msgr2=0x7f2d24009770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400a3e0 0x7f2d24009770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:43.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 0x7f2d2400fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380c5690 con 0x7f2d24003af0 2024-04-22T07:08:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400f9b0 2024-04-22T07:08:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 msgr2=0x7f2d2400fdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:43.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.599+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 0x7f2d2400fdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:43.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.800+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:43.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.800+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 msgr2=0x7f2d2400fdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:43.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:43.800+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 0x7f2d2400fdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:44.201+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:44.201+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 msgr2=0x7f2d2400fdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:44.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:44.201+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 0x7f2d2400fdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:44.999 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:45.002+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:45.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:45.002+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 msgr2=0x7f2d2400fdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:45.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:45.002+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 0x7f2d2400fdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:46.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 msgr2=0x7f2d2400fdc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:46.597 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f9b0 0x7f2d2400fdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:46.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:46.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:46.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:08:46.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d24003af0 2024-04-22T07:08:46.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380c5690 con 0x7f2d24009ea0 2024-04-22T07:08:46.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d240112d0 2024-04-22T07:08:46.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:46.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:46.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d24010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:46.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.599+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:46.797 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.800+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.800+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.800+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d24010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:46.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:46.800+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:47.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:47.201+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:47.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:47.201+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d24010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:47.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:47.201+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:47.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:47.201+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:48.002+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:48.002+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d24010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:48.002+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:48.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:48.002+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:49.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 msgr2=0x7f2d240136c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:49.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240112d0 0x7f2d240136c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:49.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:49.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d24010300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:49.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 0x7f2d2400e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:49.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:49.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24010df0 2024-04-22T07:08:49.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24010a10 2024-04-22T07:08:49.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 msgr2=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:49.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 0x7f2d2400e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:49.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.599+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:49.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.600+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:49.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.800+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 msgr2=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:49.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.800+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 0x7f2d2400e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:49.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.801+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:49.798 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:49.801+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:50.198 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:50.201+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 msgr2=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:50.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:50.201+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 0x7f2d2400e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:50.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:50.202+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:50.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:50.202+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:51.002+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 msgr2=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:51.002+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 0x7f2d2400e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:51.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:51.003+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:51.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:51.003+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 msgr2=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24010a10 0x7f2d2400e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:52.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:52.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:52.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:08:52.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d2400e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:52.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24003af0 2024-04-22T07:08:52.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24009ea0 2024-04-22T07:08:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24010df0 2024-04-22T07:08:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d24015c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d2400e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:52.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.600+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:52.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.801+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d24015c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:52.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.801+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:52.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.801+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d2400e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:52.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.801+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:52.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:52.801+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:53.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:53.202+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:53.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:53.202+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d24015c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:53.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:53.202+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d2400e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:53.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:53.202+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:53.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:53.202+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:54.003+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d24015c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:08:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:54.003+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:54.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:54.003+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d2400e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:54.003+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:08:54.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:54.003+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:55.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:55.598 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24010df0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:55.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:55.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d2400e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:55.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24003af0 legacy=0x7f2d24015c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:55.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:55.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d24006230 2024-04-22T07:08:55.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400d560 2024-04-22T07:08:55.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006230 legacy=0x7f2d240152f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:55.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:55.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.600+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:55.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.801+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006230 legacy=0x7f2d240152f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:55.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.801+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:55.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:55.801+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:56.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:56.202+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006230 legacy=0x7f2d240152f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:56.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:56.202+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:56.199 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:56.202+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:57.000 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:57.003+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006230 legacy=0x7f2d240152f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:57.003+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:57.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:57.003+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:08:58.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:58.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:08:58.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006230 legacy=0x7f2d240152f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:08:58.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:08:58.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 0x7f2d2400d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:58.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:08:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24006230 2024-04-22T07:08:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24009ea0 2024-04-22T07:08:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24003af0 2024-04-22T07:08:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 msgr2=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006230 legacy=0x7f2d240152f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:58.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.601+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 0x7f2d2400d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:08:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.802+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:58.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.802+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.802+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 msgr2=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.802+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006230 legacy=0x7f2d240152f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:58.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:58.802+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 0x7f2d2400d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:08:59.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:59.203+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:08:59.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:59.203+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:08:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:59.203+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006230 legacy=0x7f2d240152f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:08:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:59.203+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 msgr2=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:08:59.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:08:59.203+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 0x7f2d2400d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:00.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:00.004+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:00.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:00.004+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:00.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:00.004+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006230 legacy=0x7f2d240152f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:00.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:00.004+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 msgr2=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:00.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:00.004+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 0x7f2d2400d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 msgr2=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24003af0 0x7f2d2400d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:01.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:01.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:01.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006230 legacy=0x7f2d240152f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:01.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 0x7f2d24009750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:01.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d24006b80 2024-04-22T07:09:01.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400f2a0 2024-04-22T07:09:01.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d24015a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:01.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:01.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.601+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.802+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d24015a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:01.799 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.802+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:01.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:01.802+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:02.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:02.203+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d24015a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:02.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:02.203+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:02.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:02.203+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:03.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:03.004+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d24015a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:03.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:03.004+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:03.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:03.004+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:04.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 msgr2=0x7f2d24009750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:04.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f2a0 0x7f2d24009750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:04.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d24015a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:04.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:04.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:09:04.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 0x7f2d2400d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:04.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24006b80 2024-04-22T07:09:04.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24009ea0 2024-04-22T07:09:04.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24006230 2024-04-22T07:09:04.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:04.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:04.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:04.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 msgr2=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:04.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.601+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 0x7f2d2400d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:04.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.802+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:04.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.802+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:04.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.802+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:04.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.802+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 msgr2=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:04.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:04.802+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 0x7f2d2400d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:05.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:05.203+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:05.200 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:05.203+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:05.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:05.203+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:05.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:05.203+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 msgr2=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:05.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:05.203+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 0x7f2d2400d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:06.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:06.004+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:06.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:06.004+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:06.001 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:06.004+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:06.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:06.004+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 msgr2=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:06.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:06.004+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 0x7f2d2400d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:07.599 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 msgr2=0x7f2d2400d560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24006230 0x7f2d2400d560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d24003ad0 2024-04-22T07:09:07.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24009750 2024-04-22T07:09:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24003ad0 legacy=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 msgr2=0x7f2d2400f0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:07.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.602+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:07.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.803+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 msgr2=0x7f2d2400f0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.803+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24003ad0 legacy=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:07.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:07.803+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:08.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:08.204+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24003ad0 legacy=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:08.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:08.204+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 msgr2=0x7f2d2400f0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:08.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:08.204+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:09.002 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:09.005+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 msgr2=0x7f2d2400f0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:09.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:09.005+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24003ad0 legacy=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:09.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:09.005+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:10.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 msgr2=0x7f2d2400f0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:10.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:10.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24003ad0 legacy=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:10.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:10.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:09:10.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:10.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24015710 2024-04-22T07:09:10.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24009ea0 2024-04-22T07:09:10.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24009750 2024-04-22T07:09:10.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:10.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:10.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 msgr2=0x7f2d2400f0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:10.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24015710 legacy=0x7f2d24015af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:10.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.602+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.803+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.803+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:10.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.803+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 msgr2=0x7f2d2400f0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.803+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24015710 legacy=0x7f2d24015af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:10.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:10.803+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:11.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:11.204+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:11.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:11.204+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:11.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:11.204+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 msgr2=0x7f2d2400f0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:11.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:11.204+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:11.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:11.204+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24015710 legacy=0x7f2d24015af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:11.741 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:09:11.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:09:12.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:12.005+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:12.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:12.005+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:12.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:12.005+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 msgr2=0x7f2d2400f0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:12.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:12.005+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24015710 legacy=0x7f2d24015af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:12.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:12.005+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:13.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 msgr2=0x7f2d2400f0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:13.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009750 0x7f2d2400f0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:13.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:13.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:13.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24015710 legacy=0x7f2d24015af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:13.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:13.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d24006230 2024-04-22T07:09:13.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400d560 2024-04-22T07:09:13.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006230 legacy=0x7f2d24006b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:13.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:13.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.601+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:13.800 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.802+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006230 legacy=0x7f2d24006b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:13.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.802+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:13.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:13.802+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:14.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:14.203+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006230 legacy=0x7f2d24006b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:14.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:14.203+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:14.201 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:14.203+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:15.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:15.004+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006230 legacy=0x7f2d24006b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:15.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:15.004+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:15.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:15.004+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:16.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24006230 legacy=0x7f2d24006b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:09:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24016740 2024-04-22T07:09:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24009ea0 2024-04-22T07:09:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d2400d560 2024-04-22T07:09:16.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:16.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:16.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:16.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:16.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.602+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:16.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.803+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:16.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.803+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:16.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.803+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:16.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.803+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:16.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:16.803+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:17.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:17.204+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:17.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:17.204+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:17.204+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:17.204+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:17.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:17.204+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:18.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:18.005+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:18.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:18.005+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:18.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:18.005+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:18.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:18.005+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:18.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:18.005+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:19.600 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 msgr2=0x7f2d24016200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d2400d560 0x7f2d24016200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009ea0 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:19.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d24006b80 2024-04-22T07:09:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400f0b0 2024-04-22T07:09:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d24015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:19.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.602+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:19.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.803+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:19.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.803+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d24015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:19.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:19.803+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:20.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:20.204+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:20.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:20.204+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d24015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:20.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:20.204+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:21.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:21.005+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d24015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:21.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:21.005+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:21.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:21.005+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:22.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d24015710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:22.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:22.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24006b80 legacy=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:22.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:22.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:09:22.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24016740 2024-04-22T07:09:22.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d24015c50 2024-04-22T07:09:22.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400f0b0 2024-04-22T07:09:22.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:22.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d24015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:22.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24015c50 legacy=0x7f2d2401ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:22.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.602+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.803+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.803+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24015c50 legacy=0x7f2d2401ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:22.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.803+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d24015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:22.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:22.803+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:23.204+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:23.204+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24015c50 legacy=0x7f2d2401ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:23.204+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d24015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:23.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:23.204+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:24.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:24.005+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:24.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:24.005+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d24015710 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:24.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:24.005+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24015c50 legacy=0x7f2d2401ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:24.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:24.005+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:25.601 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d24015710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d24015710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24015c50 legacy=0x7f2d2401ae40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 0x7f2d24016030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400d560 2024-04-22T07:09:25.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24009ea0 2024-04-22T07:09:25.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:25.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:25.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 msgr2=0x7f2d24016030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:25.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.603+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 0x7f2d24016030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:25.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.804+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:25.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.804+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:25.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.804+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 msgr2=0x7f2d24016030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:25.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:25.804+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 0x7f2d24016030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:26.205+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:26.205+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:26.205+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 msgr2=0x7f2d24016030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:26.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:26.205+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 0x7f2d24016030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:27.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:27.006+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:27.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:27.006+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:27.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:27.006+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 msgr2=0x7f2d24016030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:27.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:27.006+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 0x7f2d24016030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 msgr2=0x7f2d24016030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24009ea0 0x7f2d24016030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:28.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:09:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380c5690 con 0x7f2d24016740 2024-04-22T07:09:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400d560 2024-04-22T07:09:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d2400f0b0 2024-04-22T07:09:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.603+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.604+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:28.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.604+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:28.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.604+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:28.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.604+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:28.801 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.804+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:28.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.805+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.805+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.805+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:28.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:28.805+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:29.202 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:29.205+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:29.203 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:29.206+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:29.206+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:29.206+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:29.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:29.206+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:30.003 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:30.006+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:30.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:30.007+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:30.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:30.007+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:30.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:30.007+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:30.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:30.007+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:31.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:31.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:31.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:31.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:31.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:31.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:31.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24009ea0 2024-04-22T07:09:31.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24008e60 2024-04-22T07:09:31.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:31.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:31.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.604+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:31.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.805+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:31.802 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.805+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:31.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:31.805+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:32.206+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:32.206+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:32.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:32.206+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:33.004 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:33.007+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:33.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:33.007+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:33.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:33.007+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:34.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:34.602 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:09:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380c5690 con 0x7f2d24016740 2024-04-22T07:09:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400f0b0 2024-04-22T07:09:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24008e60 2024-04-22T07:09:34.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:34.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.604+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:34.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.805+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:34.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.805+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:34.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.805+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:34.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.805+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:34.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:34.805+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:35.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:35.206+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:35.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:35.206+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:35.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:35.206+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:35.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:35.206+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:35.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:35.206+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:36.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:36.007+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:36.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:36.007+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:36.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:36.007+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:36.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:36.007+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:36.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:36.007+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:37.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.604+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:37.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.604+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:37.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.604+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:37.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.604+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:37.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.604+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:37.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.604+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:37.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.604+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24009ea0 2024-04-22T07:09:37.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.604+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400d560 2024-04-22T07:09:37.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.605+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:37.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.605+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:37.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.605+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:37.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.806+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:37.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.806+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:37.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:37.806+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:38.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:38.207+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:38.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:38.207+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:38.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:38.207+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:39.005 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:39.008+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:39.008+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:39.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:39.008+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:40.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:40.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:40.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:09:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380c5690 con 0x7f2d24009ea0 2024-04-22T07:09:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400d560 2024-04-22T07:09:40.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d2400f0b0 2024-04-22T07:09:40.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:40.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:40.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:40.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:40.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.605+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.806+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.806+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.806+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.806+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:40.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:40.806+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:41.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:41.207+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:41.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:41.207+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:41.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:41.207+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:41.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:41.207+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:41.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:41.207+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:41.797 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:09:41.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:09:42.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:42.008+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:42.008+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:42.008+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:42.008+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:42.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:42.008+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:43.603 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:43.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400f0b0 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:43.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:43.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:43.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:43.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:43.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24016740 2024-04-22T07:09:43.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24008e60 2024-04-22T07:09:43.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:43.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:43.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.605+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:43.803 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.806+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:43.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.806+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:43.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:43.806+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:44.204 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:44.207+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:44.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:44.207+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:44.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:44.207+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:45.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:45.008+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:45.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:45.008+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:45.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:45.008+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:46.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 msgr2=0x7f2d24009240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:46.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24008e60 0x7f2d24009240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24016740 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:09:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380c5690 con 0x7f2d24009ea0 2024-04-22T07:09:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:46.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400d560 2024-04-22T07:09:46.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24016740 2024-04-22T07:09:46.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:46.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:46.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:46.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.606+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:46.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.807+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.807+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.807+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.807+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:46.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:46.807+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:47.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:47.208+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:47.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:47.208+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:47.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:47.208+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:47.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:47.208+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:47.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:47.208+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:48.009+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:48.009+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:48.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:48.009+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:48.009+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:48.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:48.009+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:49.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:49.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:49.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:49.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:49.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:49.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 0x7f2d24009270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:49.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d2400eee0 2024-04-22T07:09:49.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24008e60 2024-04-22T07:09:49.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 msgr2=0x7f2d24009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:49.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400eee0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:49.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.606+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 0x7f2d24009270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:49.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.807+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400eee0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:49.804 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.807+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 msgr2=0x7f2d24009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:49.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:49.807+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 0x7f2d24009270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:50.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:50.208+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400eee0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:50.205 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:50.208+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 msgr2=0x7f2d24009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:50.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:50.208+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 0x7f2d24009270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:51.009+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 msgr2=0x7f2d24009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:51.009+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400eee0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:51.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:51.009+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 0x7f2d24009270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:52.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 msgr2=0x7f2d24009270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:52.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24008e60 0x7f2d24009270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:52.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400eee0 legacy=0x7f2d24008750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:52.604 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:52.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:09:52.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380c5690 con 0x7f2d24009ea0 2024-04-22T07:09:52.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d240097b0 2024-04-22T07:09:52.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24016740 2024-04-22T07:09:52.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:52.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:52.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240097b0 legacy=0x7f2d2401ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:52.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.606+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:52.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.807+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:52.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.807+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240097b0 legacy=0x7f2d2401ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:52.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.807+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:52.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:52.807+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:53.208+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:53.208+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240097b0 legacy=0x7f2d2401ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:53.208+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:53.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:53.208+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:54.006 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:54.009+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:54.009+0000 7f2d3ea1e640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:54.009+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240097b0 legacy=0x7f2d2401ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:54.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:54.009+0000 7f2d3ea1e640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240097b0 legacy=0x7f2d2401ae40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 0x7f2d24008bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:55.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24008750 2024-04-22T07:09:55.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d2400d560 2024-04-22T07:09:55.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 msgr2=0x7f2d24008bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:55.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 0x7f2d24008bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:55.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:55.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.607+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:55.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.808+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 msgr2=0x7f2d24008bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:55.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.808+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 0x7f2d24008bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:55.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.808+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:55.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:55.808+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:56.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:56.209+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 msgr2=0x7f2d24008bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:56.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:56.209+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 0x7f2d24008bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:56.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:56.209+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:56.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:56.209+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:09:57.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:57.010+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 msgr2=0x7f2d24008bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:09:57.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:57.010+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 0x7f2d24008bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:57.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:57.010+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:09:57.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:57.010+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:09:58.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d24006610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:58.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d24006610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:58.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 msgr2=0x7f2d24008bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:09:58.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24008750 0x7f2d24008bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:09:58.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:09:58.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:09:58.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d24009ea0 2024-04-22T07:09:58.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380c5690 con 0x7f2d2400d560 2024-04-22T07:09:58.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24016740 2024-04-22T07:09:58.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:58.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:58.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:09:58.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.607+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d560 legacy=0x7f2d24008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:58.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.808+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:58.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.808+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:58.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.808+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d560 legacy=0x7f2d24008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:58.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:58.808+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:09:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:59.209+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:09:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:59.209+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d560 legacy=0x7f2d24008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:09:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:59.209+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:09:59.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:09:59.209+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:00.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:00.010+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:00.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:00.010+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:00.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:00.010+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d560 legacy=0x7f2d24008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:00.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:00.010+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:01.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d2400e9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:01.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d2400e9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:01.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d560 legacy=0x7f2d24008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:01.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:01.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 0x7f2d24008e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:01.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:01.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24006610 2024-04-22T07:10:01.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24009380 2024-04-22T07:10:01.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:01.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:01.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 msgr2=0x7f2d24016bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:01.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.607+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:01.805 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.808+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.808+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.808+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 msgr2=0x7f2d24016bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:01.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:01.808+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:02.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:02.209+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:02.206 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:02.209+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:02.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:02.209+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 msgr2=0x7f2d24016bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:02.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:02.209+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:03.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:03.010+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:03.007 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:03.010+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:03.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:03.010+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 msgr2=0x7f2d24016bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:03.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:03.010+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:04.605 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 msgr2=0x7f2d24016bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:04.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:04.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:04.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24006610 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:04.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 0x7f2d24019140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:04.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:04.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:10:04.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24009ea0 2024-04-22T07:10:04.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24009760 2024-04-22T07:10:04.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24009380 2024-04-22T07:10:04.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 msgr2=0x7f2d24019140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:04.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 0x7f2d24019140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:04.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:04.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 msgr2=0x7f2d24016bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:04.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.608+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:04.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.809+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 msgr2=0x7f2d24019140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:04.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.809+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:04.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.809+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 0x7f2d24019140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:04.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.809+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 msgr2=0x7f2d24016bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:04.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:04.809+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:05.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:05.210+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 msgr2=0x7f2d24019140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:05.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:05.210+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:05.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:05.210+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 0x7f2d24019140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:05.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:05.210+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 msgr2=0x7f2d24016bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:05.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:05.210+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:06.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:06.011+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:06.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:06.011+0000 7f2d3e21d640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 msgr2=0x7f2d24019140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:06.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:06.011+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 0x7f2d24019140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:06.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:06.011+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 msgr2=0x7f2d24016bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:06.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:06.011+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:07.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 msgr2=0x7f2d24016bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:07.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24009380 0x7f2d24016bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:07.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 msgr2=0x7f2d24019140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:07.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d24009760 0x7f2d24019140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:07.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:07.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:07.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d2400d560 2024-04-22T07:10:07.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24016740 2024-04-22T07:10:07.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d560 legacy=0x7f2d24008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:07.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:07.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.608+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.809+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d560 legacy=0x7f2d24008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.809+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:07.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:07.809+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:08.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:08.210+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d560 legacy=0x7f2d24008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:08.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:08.210+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:08.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:08.210+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:09.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:09.011+0000 7f2d3da1c640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:09.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:09.011+0000 7f2d3da1c640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:09.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:09.011+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d560 legacy=0x7f2d24008730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:10.606 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:10.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:10.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d2400d560 legacy=0x7f2d24008730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:10.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:10.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:10:10.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:10.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24009ea0 2024-04-22T07:10:10.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400d560 2024-04-22T07:10:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24016740 2024-04-22T07:10:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:10.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.608+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:10.806 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.809+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:10.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.809+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:10.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.809+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:10.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.809+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:10.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:10.809+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:11.210+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:11.210+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:11.210+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:11.207 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:11.210+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:11.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:11.210+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:12.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:12.011+0000 7f2d3da1c640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:12.008 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:12.011+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:12.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:12.011+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:12.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:12.011+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:12.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:12.011+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:12.037 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:12.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:13.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:13.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:13.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:13.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:13.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:13.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:13.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d240086c0 2024-04-22T07:10:13.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d240146c0 2024-04-22T07:10:13.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 msgr2=0x7f2d24014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:13.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:13.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.609+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240086c0 legacy=0x7f2d24009380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:13.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.810+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 msgr2=0x7f2d24014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:13.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.810+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:13.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:13.810+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240086c0 legacy=0x7f2d24009380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:14.211+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 msgr2=0x7f2d24014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:14.211+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:14.208 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:14.211+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240086c0 legacy=0x7f2d24009380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:15.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:15.011+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 msgr2=0x7f2d24014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:15.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:15.011+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:15.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:15.011+0000 7f2d3e21d640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240086c0 legacy=0x7f2d24009380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:16.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 msgr2=0x7f2d24014ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:16.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:16.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240086c0 legacy=0x7f2d24009380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:16.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:16.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:10:16.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:16.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24009ea0 2024-04-22T07:10:16.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d240146c0 2024-04-22T07:10:16.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24016740 2024-04-22T07:10:16.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 msgr2=0x7f2d24014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:16.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:16.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:16.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:16.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.608+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:16.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.809+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 msgr2=0x7f2d24014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:16.807 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.809+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:16.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.809+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:16.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.809+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:16.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:16.809+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:17.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:17.210+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 msgr2=0x7f2d24014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:17.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:17.210+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:17.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:17.210+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:17.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:17.210+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:17.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:17.210+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:18.009 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:18.011+0000 7f2d3ea1e640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:18.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:18.011+0000 7f2d3da1c640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 msgr2=0x7f2d24014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:18.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:18.011+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:18.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:18.011+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:18.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:18.011+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:19.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:19.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:19.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 msgr2=0x7f2d24014ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:19.607 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d240146c0 0x7f2d24014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:19.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d37fff640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:19.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:19.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d240086c0 2024-04-22T07:10:19.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400d560 2024-04-22T07:10:19.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240086c0 legacy=0x7f2d24008fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:19.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:19.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.609+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:19.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.810+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:19.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.810+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240086c0 legacy=0x7f2d24008fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:19.809 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:19.810+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:20.211+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:20.211+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240086c0 legacy=0x7f2d24008fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:20.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:20.211+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:21.012+0000 7f2d3e21d640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:21.010 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:21.012+0000 7f2d3e21d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:21.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:21.012+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240086c0 legacy=0x7f2d24008fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:22.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d37fff640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:22.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d2400d560 0x7f2d2400e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:22.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d240086c0 legacy=0x7f2d24008fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:22.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:22.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d3ea1e640 1 -- reap_dead start 2024-04-22T07:10:22.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d2400e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:22.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d37fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2400a3e0 con 0x7f2d24009ea0 2024-04-22T07:10:22.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d37fff640 1 -- --> v2:172.21.15.44:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d2400d560 2024-04-22T07:10:22.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d37fff640 1 -- --> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380c53f0 con 0x7f2d24016740 2024-04-22T07:10:22.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:22.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:22.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:22.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:22.610 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.609+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d2400e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.810+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.810+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d2400e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.810+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.810+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:22.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:22.810+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:23.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:23.211+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:23.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:23.211+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d2400e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:23.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:23.211+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:23.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:23.211+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:23.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:23.211+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:24.012+0000 7f2d3ea1e640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.44:3300/0 2024-04-22T07:10:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:24.012+0000 7f2d3ea1e640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d2400e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:24.012+0000 7f2d3da1c640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.70:6789/0 2024-04-22T07:10:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:24.012+0000 7f2d3e21d640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3301/0 2024-04-22T07:10:24.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:24.012+0000 7f2d3e21d640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d37fff640 1 -- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 msgr2=0x7f2d24008aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d37fff640 1 --2- >> [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] conn(0x7f2d24016740 0x7f2d24008aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d37fff640 1 -- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 msgr2=0x7f2d2400e250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d37fff640 1 --2- >> v2:172.21.15.44:3300/0 conn(0x7f2d2400d560 0x7f2d2400e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T07:10:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d37fff640 1 -- >> v1:172.21.15.70:6789/0 conn(0x7f2d24009ea0 legacy=0x7f2d24008fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T07:10:25.608 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24007dc0 0x7f2d240081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T07:10:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d37fff640 1 -- --> v1:172.21.15.44:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d380718a0 con 0x7f2d2400e790 2024-04-22T07:10:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d37fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 -- 0x7f2d38157db0 con 0x7f2d24007dc0 2024-04-22T07:10:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d2400e790 legacy=0x7f2d240076b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24007dc0 msgr2=0x7f2d240081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:25.609 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.609+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24007dc0 0x7f2d240081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T07:10:25.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.810+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d2400e790 legacy=0x7f2d240076b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:25.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.810+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24007dc0 msgr2=0x7f2d240081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:25.808 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:25.810+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24007dc0 0x7f2d240081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T07:10:26.209 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:26.211+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d2400e790 legacy=0x7f2d240076b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:26.211+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24007dc0 msgr2=0x7f2d240081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:26.210 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:26.211+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24007dc0 0x7f2d240081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T07:10:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:27.012+0000 7f2d3e21d640 1 -- >> v1:172.21.15.44:6789/0 conn(0x7f2d2400e790 legacy=0x7f2d240076b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.44:6789/0 2024-04-22T07:10:27.011 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:27.012+0000 7f2d3da1c640 1 -- >> v2:172.21.15.70:3300/0 conn(0x7f2d24007dc0 msgr2=0x7f2d240081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.70:3300/0 2024-04-22T07:10:27.012 INFO:teuthology.orchestra.run.smithi044.stderr:2024-04-22T07:10:27.012+0000 7f2d3da1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d24007dc0 0x7f2d240081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T07:10:28.519 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-22T07:10:28.520 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-22T07:10:28.541 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-22T07:10:28.541 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-22T07:10:28.541 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.545 INFO:tasks.ceph.mds.a:Stopped 2024-04-22T07:10:28.546 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-22T07:10:28.546 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:28.546 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c' 2024-04-22T07:10:28.563 INFO:tasks.ceph.mds.c:Stopped 2024-04-22T07:10:28.563 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-04-22T07:10:28.563 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:28.564 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e' 2024-04-22T07:10:28.582 INFO:tasks.ceph.mds.e:Stopped 2024-04-22T07:10:28.582 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-22T07:10:28.583 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:28.583 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi070 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2024-04-22T07:10:28.603 INFO:tasks.ceph.mds.b:Stopped 2024-04-22T07:10:28.603 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-22T07:10:28.603 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:28.603 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi070 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2024-04-22T07:10:28.622 INFO:tasks.ceph.mds.d:Stopped 2024-04-22T07:10:28.622 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-04-22T07:10:28.622 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.622 INFO:tasks.ceph.mds.f:Stopped 2024-04-22T07:10:28.622 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-22T07:10:28.623 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-22T07:10:28.623 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.623 INFO:tasks.ceph.osd.0:Stopped 2024-04-22T07:10:28.623 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-22T07:10:28.623 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.623 INFO:tasks.ceph.osd.1:Stopped 2024-04-22T07:10:28.624 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-22T07:10:28.624 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.624 INFO:tasks.ceph.osd.2:Stopped 2024-04-22T07:10:28.624 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-22T07:10:28.624 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.624 INFO:tasks.ceph.osd.3:Stopped 2024-04-22T07:10:28.624 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-22T07:10:28.624 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.625 INFO:tasks.ceph.osd.4:Stopped 2024-04-22T07:10:28.625 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-22T07:10:28.625 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.625 INFO:tasks.ceph.osd.5:Stopped 2024-04-22T07:10:28.626 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-22T07:10:28.626 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.626 INFO:tasks.ceph.osd.6:Stopped 2024-04-22T07:10:28.626 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-22T07:10:28.626 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.626 INFO:tasks.ceph.osd.7:Stopped 2024-04-22T07:10:28.627 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-22T07:10:28.627 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-22T07:10:28.627 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.627 INFO:tasks.ceph.mgr.y:Stopped 2024-04-22T07:10:28.627 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-22T07:10:28.627 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:28.628 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi070 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' 2024-04-22T07:10:28.657 INFO:tasks.ceph.mgr.x:Stopped 2024-04-22T07:10:28.657 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-22T07:10:28.658 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-22T07:10:28.658 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.658 INFO:tasks.ceph.mon.a:Stopped 2024-04-22T07:10:28.658 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-22T07:10:28.658 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.659 INFO:tasks.ceph.mon.b:Stopped 2024-04-22T07:10:28.659 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-22T07:10:28.659 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T07:10:28.659 INFO:tasks.ceph.mon.c:Stopped 2024-04-22T07:10:28.659 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-22T07:10:28.660 DEBUG:teuthology.orchestra.run.smithi044:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-04-22T07:10:28.720 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:10:28.720 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-22T07:10:28.809 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:10:28.809 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-22T07:10:28.869 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:10:28.869 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-22T07:10:28.926 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:10:28.927 DEBUG:teuthology.orchestra.run.smithi044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-22T07:10:28.983 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T07:10:28.983 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-22T07:10:29.098 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T07:10:29.099 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-22T07:10:29.156 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T07:10:29.157 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-22T07:10:29.206 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T07:10:29.206 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-22T07:10:29.264 INFO:tasks.ceph:Archiving mon data... 2024-04-22T07:10:29.266 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157/data/mon.a.tgz 2024-04-22T07:10:29.266 DEBUG:teuthology.orchestra.run.smithi044:> mktemp 2024-04-22T07:10:29.282 INFO:teuthology.orchestra.run.smithi044.stdout:/tmp/tmp.g9uHgym7xQ 2024-04-22T07:10:29.282 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.g9uHgym7xQ 2024-04-22T07:10:29.433 DEBUG:teuthology.orchestra.run.smithi044:> sudo chmod 0666 /tmp/tmp.g9uHgym7xQ 2024-04-22T07:10:29.516 DEBUG:teuthology.orchestra.remote:smithi044:/tmp/tmp.g9uHgym7xQ is 269KB 2024-04-22T07:10:29.611 DEBUG:teuthology.orchestra.run.smithi044:> rm -fr /tmp/tmp.g9uHgym7xQ 2024-04-22T07:10:29.626 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157/data/mon.b.tgz 2024-04-22T07:10:29.626 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-04-22T07:10:29.644 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.H4fopwtAvb 2024-04-22T07:10:29.644 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.H4fopwtAvb 2024-04-22T07:10:29.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.H4fopwtAvb 2024-04-22T07:10:29.866 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.H4fopwtAvb is 360KB 2024-04-22T07:10:29.933 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.H4fopwtAvb 2024-04-22T07:10:29.947 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157/data/mon.c.tgz 2024-04-22T07:10:29.947 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-04-22T07:10:30.000 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.r7IjAR9CRg 2024-04-22T07:10:30.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.r7IjAR9CRg 2024-04-22T07:10:30.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.r7IjAR9CRg 2024-04-22T07:10:30.207 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.r7IjAR9CRg is 360KB 2024-04-22T07:10:30.302 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.r7IjAR9CRg 2024-04-22T07:10:30.316 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-22T07:10:30.316 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-22T07:10:30.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-22T07:10:30.400 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-22T07:10:30.433 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-22T07:10:30.433 INFO:tasks.ceph:Archiving crash dumps... 2024-04-22T07:10:30.436 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157/remote/smithi044/crash 2024-04-22T07:10:30.437 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-22T07:10:30.494 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157/remote/smithi070/crash 2024-04-22T07:10:30.495 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-22T07:10:30.539 INFO:tasks.ceph:Shutting down logrotate 2024-04-22T07:10:30.540 DEBUG:teuthology.orchestra.run.smithi044:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:30.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:30.620 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:30.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-22T07:10:30.668 INFO:tasks.ceph:Compressing logs... 2024-04-22T07:10:30.669 DEBUG:teuthology.orchestra.run.smithi044:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T07:10:30.689 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T07:10:30.756 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-22T07:10:30.757 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-04-22T07:10:30.757 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33550.log 2024-04-22T07:10:30.757 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-22T07:10:30.757 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33550.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-22T07:10:30.757 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33550.log.gz 2024-04-22T07:10:30.757 INFO:teuthology.orchestra.run.smithi070.stderr: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-22T07:10:30.757 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-22T07:10:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-22T07:10:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33419.log 2024-04-22T07:10:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33459.log 2024-04-22T07:10:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33419.log: gzip -5 --verbose 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-22T07:10:30.758 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-osd.4.log 2024-04-22T07:10:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33459.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33419.log.gz 2024-04-22T07:10:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33283.log 2024-04-22T07:10:30.759 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33459.log.gz 2024-04-22T07:10:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33499.log 2024-04-22T07:10:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33283.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-22T07:10:30.759 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33283.log.gz 2024-04-22T07:10:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33243.log 2024-04-22T07:10:30.759 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33499.log.gz 2024-04-22T07:10:30.760 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-22T07:10:30.760 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33379.log 86.9% 2024-04-22T07:10:30.760 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33243.log.gz 2024-04-22T07:10:30.760 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.c.log: 92.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-22T07:10:30.760 INFO:teuthology.orchestra.run.smithi070.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-22T07:10:30.760 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-22T07:10:30.760 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.33379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33339.log 2024-04-22T07:10:30.760 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33379.log.gz/var/log/ceph/ceph-osd.6.log: 2024-04-22T07:10:30.760 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-22T07:10:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-22T07:10:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.33339.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33339.log.gz 2024-04-22T07:10:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-04-22T07:10:30.762 INFO:teuthology.orchestra.run.smithi070.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-22T07:10:30.783 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41819.log 2024-04-22T07:10:30.783 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41189.log 2024-04-22T07:10:30.783 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42437.log 2024-04-22T07:10:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41189.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41819.log.gz 2024-04-22T07:10:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41903.log 2024-04-22T07:10:30.784 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41189.log.gz 2024-04-22T07:10:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41693.log 2024-04-22T07:10:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40614.log 2024-04-22T07:10:30.784 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42437.log.gz 2024-04-22T07:10:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41693.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41903.log.gz 2024-04-22T07:10:30.784 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39971.log 2024-04-22T07:10:30.784 INFO:teuthology.orchestra.run.smithi044.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41693.log.gz 2024-04-22T07:10:30.785 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41609.log 2024-04-22T07:10:30.785 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44708.log 2024-04-22T07:10:30.785 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40614.log.gz 2024-04-22T07:10:30.785 INFO:teuthology.orchestra.run.smithi044.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39971.log.gz 2024-04-22T07:10:30.785 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39799.log 2024-04-22T07:10:30.785 INFO:teuthology.orchestra.run.smithi044.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41609.log.gz 2024-04-22T07:10:30.785 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41507.log 2024-04-22T07:10:30.785 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44708.log.gz 2024-04-22T07:10:30.786 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43435.log 2024-04-22T07:10:30.786 INFO:teuthology.orchestra.run.smithi044.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39799.log.gz 2024-04-22T07:10:30.786 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41567.log 2024-04-22T07:10:30.786 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41507.log.gz 2024-04-22T07:10:30.786 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40951.log 2024-04-22T07:10:30.786 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41567.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43435.log.gz 2024-04-22T07:10:30.786 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43225.log 2024-04-22T07:10:30.786 INFO:teuthology.orchestra.run.smithi044.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41567.log.gz 2024-04-22T07:10:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40139.log 2024-04-22T07:10:30.787 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40951.log.gz 2024-04-22T07:10:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41861.log 2024-04-22T07:10:30.787 INFO:teuthology.orchestra.run.smithi044.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43225.log.gz 2024-04-22T07:10:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43519.log 2024-04-22T07:10:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41861.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40139.log.gz 2024-04-22T07:10:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-22T07:10:30.787 INFO:teuthology.orchestra.run.smithi044.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41861.log.gz 2024-04-22T07:10:30.787 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40013.log 2024-04-22T07:10:30.788 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43519.log.gz 2024-04-22T07:10:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43477.log 2024-04-22T07:10:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41374.log 2024-04-22T07:10:30.788 INFO:teuthology.orchestra.run.smithi044.stderr: 84.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-22T07:10:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43477.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40013.log.gz 2024-04-22T07:10:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-22T07:10:30.788 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43477.log.gz 2024-04-22T07:10:30.788 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40615.log 2024-04-22T07:10:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33212.log 2024-04-22T07:10:30.789 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41374.log.gz 2024-04-22T07:10:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39843.log 2024-04-22T07:10:30.789 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40615.log.gz 2024-04-22T07:10:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.33212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43057.log 2024-04-22T07:10:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39843.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-22T07:10:30.789 INFO:teuthology.orchestra.run.smithi044.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.33212.log.gz 2024-04-22T07:10:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43057.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39843.log.gz 2024-04-22T07:10:30.789 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44624.log 2024-04-22T07:10:30.790 INFO:teuthology.orchestra.run.smithi044.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43057.log.gz 2024-04-22T07:10:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44750.log 2024-04-22T07:10:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44500.log 2024-04-22T07:10:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44624.log: /var/log/ceph/ceph-client.admin.44750.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44624.log.gz 2024-04-22T07:10:30.790 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44068.log 89.5% 2024-04-22T07:10:30.791 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44750.log.gz 2024-04-22T07:10:30.791 INFO:teuthology.orchestra.run.smithi044.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-22T07:10:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39569.log 2024-04-22T07:10:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44068.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44500.log.gz 2024-04-22T07:10:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44792.log 2024-04-22T07:10:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39220.log 2024-04-22T07:10:30.791 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44792.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39569.log.gz 2024-04-22T07:10:30.791 INFO:teuthology.orchestra.run.smithi044.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44068.log.gz 2024-04-22T07:10:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-22T07:10:30.792 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44792.log.gz 2024-04-22T07:10:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42203.log 2024-04-22T07:10:30.792 INFO:teuthology.orchestra.run.smithi044.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39220.log.gz 2024-04-22T07:10:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-osd.3.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44976.log 2024-04-22T07:10:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41239.log 2024-04-22T07:10:30.792 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45237.log 2024-04-22T07:10:30.792 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44976.log.gz 2024-04-22T07:10:30.792 INFO:teuthology.orchestra.run.smithi044.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.42203.log.gz 2024-04-22T07:10:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39427.log 2024-04-22T07:10:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-22T07:10:30.793 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41239.log.gz/var/log/ceph/ceph-client.admin.45237.log: 2024-04-22T07:10:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44327.log 2024-04-22T07:10:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39427.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-22T07:10:30.793 INFO:teuthology.orchestra.run.smithi044.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39427.log.gz 2024-04-22T07:10:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44327.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-22T07:10:30.793 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44327.log.gz 2024-04-22T07:10:30.793 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44543.log 2024-04-22T07:10:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:gzip/var/log/ceph/ceph-mgr.y.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40949.log 2024-04-22T07:10:30.794 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41465.log 2024-04-22T07:10:30.794 INFO:teuthology.orchestra.run.smithi044.stderr: 95.6%/var/log/ceph/ceph-client.admin.40949.log: -- replaced with /var/log/ceph/ceph-client.admin.45237.log.gz 2024-04-22T07:10:30.794 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40949.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44543.log.gz 2024-04-22T07:10:30.794 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T07:10:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-22T07:10:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43645.log 2024-04-22T07:10:30.795 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4%/var/log/ceph/ceph-osd.0.log: -- replaced with /var/log/ceph/ceph-client.admin.41465.log.gz 2024-04-22T07:10:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33476.log 2024-04-22T07:10:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40948.log 2024-04-22T07:10:30.795 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43645.log.gz 2024-04-22T07:10:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.33476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39715.log 2024-04-22T07:10:30.795 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40948.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33476.log.gz 2024-04-22T07:10:30.796 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44002.log 2024-04-22T07:10:30.796 INFO:teuthology.orchestra.run.smithi044.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40948.log.gz 2024-04-22T07:10:30.796 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5/var/log/ceph/ceph-client.admin.39715.log: --verbose -- /var/log/ceph/ceph-client.admin.40945.log 2024-04-22T07:10:30.796 INFO:teuthology.orchestra.run.smithi044.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39715.log.gz 2024-04-22T07:10:30.796 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44261.log 2024-04-22T07:10:30.796 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40945.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44002.log.gz 2024-04-22T07:10:30.796 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40945.log.gz 2024-04-22T07:10:30.797 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40223.log 2024-04-22T07:10:30.797 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40349.log 2024-04-22T07:10:30.797 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40223.log: /var/log/ceph/ceph-client.admin.44261.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40223.log.gz 2024-04-22T07:10:30.797 INFO:teuthology.orchestra.run.smithi044.stderr: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43787.log 2024-04-22T07:10:30.797 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40349.log: -- replaced with /var/log/ceph/ceph-client.admin.44261.log.gz 2024-04-22T07:10:30.797 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42029.log 2024-04-22T07:10:30.798 INFO:teuthology.orchestra.run.smithi044.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40349.log.gz 2024-04-22T07:10:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40433.log 2024-04-22T07:10:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42029.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43787.log.gz 2024-04-22T07:10:30.798 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42029.log.gz 2024-04-22T07:10:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43869.log 2024-04-22T07:10:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43351.log 2024-04-22T07:10:30.798 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40433.log: /var/log/ceph/ceph-client.admin.43869.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40433.log.gz 2024-04-22T07:10:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44850.log 2024-04-22T07:10:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43351.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43869.log.gz 2024-04-22T07:10:30.799 INFO:teuthology.orchestra.run.smithi044.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43351.log.gzgzip 2024-04-22T07:10:30.799 INFO:teuthology.orchestra.run.smithi044.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40604.log 2024-04-22T07:10:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43729.log 2024-04-22T07:10:30.799 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40604.log: /var/log/ceph/ceph-client.admin.44850.log: 90.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 88.0% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40604.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44850.log.gz 2024-04-22T07:10:30.799 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T07:10:30.799 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T07:10:30.800 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44934.log 2024-04-22T07:10:30.800 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43729.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-22T07:10:30.800 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7%/var/log/ceph/ceph-client.admin.44934.log: -- replaced with /var/log/ceph/ceph-client.admin.43729.log.gz 2024-04-22T07:10:30.800 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44934.log.gz 2024-04-22T07:10:30.800 INFO:teuthology.orchestra.run.smithi044.stderr: /var/log/ceph/ceph-client.admin.40612.log 2024-04-22T07:10:30.800 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41147.log 2024-04-22T07:10:30.800 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-04-22T07:10:30.801 INFO:teuthology.orchestra.run.smithi044.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40612.log.gz 2024-04-22T07:10:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41147.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41147.log.gz 2024-04-22T07:10:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44666.log 2024-04-22T07:10:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42071.log 2024-04-22T07:10:30.801 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43183.log: /var/log/ceph/ceph-client.admin.44666.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-04-22T07:10:30.801 INFO:teuthology.orchestra.run.smithi044.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44666.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45417.log 2024-04-22T07:10:30.802 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T07:10:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42071.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42071.log.gz 2024-04-22T07:10:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43935.log 2024-04-22T07:10:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41945.log 2024-04-22T07:10:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45417.log: /var/log/ceph/ceph-client.admin.43935.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43935.log.gz 2024-04-22T07:10:30.802 INFO:teuthology.orchestra.run.smithi044.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.45417.log.gz 2024-04-22T07:10:30.802 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44218.log 2024-04-22T07:10:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40181.log 2024-04-22T07:10:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41945.log: /var/log/ceph/ceph-client.admin.44218.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41945.log.gz 2024-04-22T07:10:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43561.log 2024-04-22T07:10:30.803 INFO:teuthology.orchestra.run.smithi044.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44218.log.gz 2024-04-22T07:10:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40181.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40181.log.gz 2024-04-22T07:10:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40391.log 2024-04-22T07:10:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40941.log 2024-04-22T07:10:30.803 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43561.log: /var/log/ceph/ceph-client.admin.40391.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43561.log.gz 2024-04-22T07:10:30.803 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40391.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43141.log 2024-04-22T07:10:30.804 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T07:10:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40941.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40941.log.gz 2024-04-22T07:10:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43267.log 2024-04-22T07:10:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42335.log 2024-04-22T07:10:30.804 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43141.log: /var/log/ceph/ceph-client.admin.43267.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43141.log.gz 2024-04-22T07:10:30.805 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43267.log.gz 2024-04-22T07:10:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-04-22T07:10:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42335.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42335.log.gz 2024-04-22T07:10:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33579.log 2024-04-22T07:10:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41420.log 2024-04-22T07:10:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-mds.e.log: /var/log/ceph/ceph-client.admin.33579.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33579.log.gz 2024-04-22T07:10:30.805 INFO:teuthology.orchestra.run.smithi044.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-22T07:10:30.805 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43015.log 2024-04-22T07:10:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40055.log 2024-04-22T07:10:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43015.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41420.log.gz 2024-04-22T07:10:30.806 INFO:teuthology.orchestra.run.smithi044.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43015.log.gz 2024-04-22T07:10:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40265.log 2024-04-22T07:10:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40055.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40055.log.gz 2024-04-22T07:10:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40307.log 2024-04-22T07:10:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43393.log 2024-04-22T07:10:30.806 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40307.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40265.log.gz 2024-04-22T07:10:30.807 INFO:teuthology.orchestra.run.smithi044.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40307.log.gz 2024-04-22T07:10:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41651.log 2024-04-22T07:10:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43393.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43393.log.gz 2024-04-22T07:10:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45018.log 2024-04-22T07:10:30.807 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43687.log 2024-04-22T07:10:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.45018.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41651.log.gz 2024-04-22T07:10:30.808 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45018.log.gz 2024-04-22T07:10:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41735.log 2024-04-22T07:10:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40613.log 2024-04-22T07:10:30.808 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41735.log: 89.6% 90.5% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-04-22T07:10:30.808 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43687.log.gz 2024-04-22T07:10:30.808 INFO:teuthology.orchestra.run.smithi044.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41735.log.gz 2024-04-22T07:10:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39929.log 2024-04-22T07:10:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43099.log 2024-04-22T07:10:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40613.log: gzip/var/log/ceph/ceph-client.admin.39929.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40609.log 2024-04-22T07:10:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43099.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39929.log.gz 2024-04-22T07:10:30.809 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40613.log.gz 2024-04-22T07:10:30.809 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43099.log.gz 2024-04-22T07:10:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44371.log 2024-04-22T07:10:30.809 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44892.log 2024-04-22T07:10:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44371.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40609.log.gz 2024-04-22T07:10:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39887.log 2024-04-22T07:10:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40097.log 2024-04-22T07:10:30.810 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44892.log: /var/log/ceph/ceph-client.admin.39887.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44371.log.gz 2024-04-22T07:10:30.810 INFO:teuthology.orchestra.run.smithi044.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44892.log.gz 2024-04-22T07:10:30.810 INFO:teuthology.orchestra.run.smithi044.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39887.log.gz 2024-04-22T07:10:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39655.log 2024-04-22T07:10:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40606.log 2024-04-22T07:10:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40097.log: gzip/var/log/ceph/ceph-client.admin.39655.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43603.log 2024-04-22T07:10:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40606.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39655.log.gz 2024-04-22T07:10:30.811 INFO:teuthology.orchestra.run.smithi044.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40097.log.gz 2024-04-22T07:10:30.811 INFO:teuthology.orchestra.run.smithi044.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40606.log.gz 2024-04-22T07:10:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41281.log 2024-04-22T07:10:30.811 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-22T07:10:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39527.log 2024-04-22T07:10:30.812 INFO:teuthology.orchestra.run.smithi044.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43603.log.gz 2024-04-22T07:10:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41281.log: /var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42269.log 2024-04-22T07:10:30.812 INFO:teuthology.orchestra.run.smithi044.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41281.log.gz 2024-04-22T07:10:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41332.log 2024-04-22T07:10:30.812 INFO:teuthology.orchestra.run.smithi044.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39527.log.gz 2024-04-22T07:10:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44134.log 2024-04-22T07:10:30.812 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41332.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42269.log.gz 2024-04-22T07:10:30.813 INFO:teuthology.orchestra.run.smithi044.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41332.log.gz 2024-04-22T07:10:30.813 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40943.log 2024-04-22T07:10:30.813 INFO:teuthology.orchestra.run.smithi044.stderr: 91.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41777.log 2024-04-22T07:10:30.813 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-22T07:10:30.813 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.44134.log: /var/log/ceph/ceph-client.admin.40943.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40943.log.gz 2024-04-22T07:10:30.814 INFO:teuthology.orchestra.run.smithi044.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44134.log.gz 2024-04-22T07:10:30.814 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39613.log 2024-04-22T07:10:30.814 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42876.log 2024-04-22T07:10:30.814 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42137.log 2024-04-22T07:10:30.814 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41777.log.gz 2024-04-22T07:10:30.814 INFO:teuthology.orchestra.run.smithi044.stderr: 88.2%/var/log/ceph/ceph-client.0.42876.log: -- replaced with /var/log/ceph/ceph-client.admin.39613.log.gz 2024-04-22T07:10:30.814 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40946.log 2024-04-22T07:10:30.815 INFO:teuthology.orchestra.run.smithi070.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-22T07:10:30.819 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.42137.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.42137.log.gz 2024-04-22T07:10:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41987.log 2024-04-22T07:10:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39757.log 2024-04-22T07:10:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.41987.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40946.log.gz 2024-04-22T07:10:30.820 INFO:teuthology.orchestra.run.smithi044.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41987.log.gz 2024-04-22T07:10:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39312.log 2024-04-22T07:10:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39757.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39757.log.gz 2024-04-22T07:10:30.820 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43309.log 2024-04-22T07:10:30.820 INFO:teuthology.orchestra.run.smithi044.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.42876.log.gz 2024-04-22T07:10:30.821 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.39312.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27185.log 2024-04-22T07:10:30.821 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.43309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40931.log 2024-04-22T07:10:30.821 INFO:teuthology.orchestra.run.smithi044.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39312.log.gz 2024-04-22T07:10:30.821 INFO:teuthology.orchestra.run.smithi044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40610.log 2024-04-22T07:10:30.821 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph.tmp-client.admin.27185.log: 88.5% 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27185.log.gz 2024-04-22T07:10:30.821 INFO:teuthology.orchestra.run.smithi044.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43309.log.gz 2024-04-22T07:10:30.821 INFO:teuthology.orchestra.run.smithi044.stderr:/var/log/ceph/ceph-client.admin.40931.log: 88.6%/var/log/ceph/ceph-client.admin.40610.log: -- replaced with /var/log/ceph/ceph-client.admin.40931.log.gz 2024-04-22T07:10:30.821 INFO:teuthology.orchestra.run.smithi044.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40610.log.gz 2024-04-22T07:10:30.822 INFO:teuthology.orchestra.run.smithi044.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-22T07:10:30.822 INFO:teuthology.orchestra.run.smithi044.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-22T07:10:30.823 INFO:teuthology.orchestra.run.smithi070.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-22T07:10:30.828 INFO:teuthology.orchestra.run.smithi070.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-22T07:10:30.828 INFO:teuthology.orchestra.run.smithi070.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-22T07:10:30.836 INFO:teuthology.orchestra.run.smithi044.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-22T07:10:30.846 INFO:teuthology.orchestra.run.smithi044.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-22T07:10:30.847 INFO:teuthology.orchestra.run.smithi070.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-22T07:10:30.871 INFO:teuthology.orchestra.run.smithi070.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-22T07:10:30.873 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-04-22T07:10:30.873 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m0.151s 2024-04-22T07:10:30.873 INFO:teuthology.orchestra.run.smithi070.stderr:user 0m0.610s 2024-04-22T07:10:30.874 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m0.034s 2024-04-22T07:10:30.891 INFO:teuthology.orchestra.run.smithi044.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-22T07:10:30.893 INFO:teuthology.orchestra.run.smithi044.stderr: 2024-04-22T07:10:30.893 INFO:teuthology.orchestra.run.smithi044.stderr:real 0m0.192s 2024-04-22T07:10:30.893 INFO:teuthology.orchestra.run.smithi044.stderr:user 0m0.683s 2024-04-22T07:10:30.894 INFO:teuthology.orchestra.run.smithi044.stderr:sys 0m0.111s 2024-04-22T07:10:30.894 INFO:tasks.ceph:Archiving logs... 2024-04-22T07:10:30.894 DEBUG:teuthology.misc:Transferring archived files from smithi044:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157/remote/smithi044/log 2024-04-22T07:10:30.895 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T07:10:31.148 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157/remote/smithi070/log 2024-04-22T07:10:31.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T07:10:31.265 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1918, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph.py", line 1928, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2822, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2654, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 2420, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1656, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-04-22T07:10:31.293 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-22T07:10:31.316 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-22T07:10:31.337 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-22T07:10:31.338 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T07:10:31.342 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T07:10:31.431 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-22T07:10:31.441 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-22T07:10:31.441 DEBUG:teuthology.orchestra.run.smithi044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:10:31.444 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi044.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-04-22T07:10:31.460 INFO:teuthology.orchestra.run.smithi044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:10:31.461 INFO:teuthology.orchestra.run.smithi044.stdout:=============================================================================== 2024-04-22T07:10:31.461 INFO:teuthology.orchestra.run.smithi044.stdout:^- hv01.front.sepia.ceph.com 2 6 377 55 +1201us[+1200us] +/- 590ms 2024-04-22T07:10:31.461 INFO:teuthology.orchestra.run.smithi044.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 -43us[ -44us] +/- 63ms 2024-04-22T07:10:31.461 INFO:teuthology.orchestra.run.smithi044.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 56 +53us[ +52us] +/- 66ms 2024-04-22T07:10:31.461 INFO:teuthology.orchestra.run.smithi044.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:10:31.462 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T07:10:31.463 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-04-22T07:10:31.463 INFO:teuthology.orchestra.run.smithi070.stdout:^- hv01.front.sepia.ceph.com 2 6 377 54 +1238us[+1251us] +/- 590ms 2024-04-22T07:10:31.463 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -9617ns[+2939ns] +/- 63ms 2024-04-22T07:10:31.463 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +83us[ +83us] +/- 66ms 2024-04-22T07:10:31.463 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T07:10:31.463 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-22T07:10:31.473 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-22T07:10:31.473 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-22T07:10:31.482 DEBUG:teuthology.orchestra.run.smithi044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T07:10:31.566 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T07:10:31.622 DEBUG:teuthology.orchestra.run.smithi044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T07:10:31.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:31.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T07:10:31.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:31.674 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-22T07:10:31.684 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-22T07:10:31.693 INFO:teuthology.task.internal:Duration was 789.328485 seconds 2024-04-22T07:10:31.693 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-22T07:10:31.703 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-22T07:10:31.703 DEBUG:teuthology.orchestra.run.smithi044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T07:10:31.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T07:10:31.741 INFO:teuthology.orchestra.run.smithi044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T07:10:31.751 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T07:10:32.109 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-22T07:10:32.110 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi044.front.sepia.ceph.com 2024-04-22T07:10:32.110 DEBUG:teuthology.orchestra.run.smithi044:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T07:10:32.170 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-04-22T07:10:32.170 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T07:10:32.196 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-22T07:10:32.196 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T07:10:32.213 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T07:10:32.261 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-22T07:10:32.261 DEBUG:teuthology.orchestra.run.smithi044:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:10:32.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:10:32.394 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-22T07:10:32.405 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-22T07:10:32.405 DEBUG:teuthology.orchestra.run.smithi044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T07:10:32.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T07:10:32.452 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-22T07:10:32.462 DEBUG:teuthology.orchestra.run.smithi044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:32.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:32.492 INFO:teuthology.orchestra.run.smithi044.stdout:kernel.core_pattern = core 2024-04-22T07:10:32.515 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-04-22T07:10:32.534 DEBUG:teuthology.orchestra.run.smithi044:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:32.566 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi044.front.sepia.ceph.com, flagging run as failed 2024-04-22T07:10:32.566 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:32.588 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi070.front.sepia.ceph.com, flagging run as failed 2024-04-22T07:10:32.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-22T07:10:32.598 INFO:teuthology.task.internal:Transferring archived files... 2024-04-22T07:10:32.599 DEBUG:teuthology.misc:Transferring archived files from smithi044:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157/remote/smithi044 2024-04-22T07:10:32.599 DEBUG:teuthology.orchestra.run.smithi044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T07:10:48.163 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2024-04-22T07:10:48.174 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157/remote/smithi044/coredump/1713769536.39429.core.gz: gzip compressed data, was "1713769536.39429.core", last modified: Mon Apr 22 07:10:40 2024, max compression, original size modulo 2^32 261820416 2024-04-22T07:10:48.174 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2024-04-22T07:10:49.168 INFO:teuthology.task.internal: core looks like: /tmp/tmpv6i2o_l2: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i c', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-04-22T07:10:49.190 INFO:teuthology.task.internal: dump_program: ceph-mds -f --cluster ceph -i c 2024-04-22T07:10:49.191 DEBUG:teuthology.orchestra.run.smithi044:> which 'ceph-mds -f --cluster ceph -i c' 2024-04-22T07:10:49.219 INFO:teuthology.orchestra.run.smithi044.stderr:/usr/bin/which: invalid option -- 'f' 2024-04-22T07:10:49.220 INFO:teuthology.orchestra.run.smithi044.stderr:/usr/bin/which: unrecognized option '--cluster' 2024-04-22T07:10:49.220 INFO:teuthology.orchestra.run.smithi044.stderr:/usr/bin/which: no c in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-04-22T07:10:49.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:49.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph-mds 2024-04-22T07:10:49.221 INFO:teuthology.orchestra.run.smithi044.stdout:/usr/bin/ceph 2024-04-22T07:10:49.221 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 1: "which 'ceph-mds -f --cluster ceph -i c'" 2024-04-22T07:10:49.239 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-22T07:10:49.250 INFO:teuthology.task.internal:Not uploading archives. 2024-04-22T07:10:49.250 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-22T07:10:49.260 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-22T07:10:49.261 DEBUG:teuthology.orchestra.run.smithi044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T07:10:49.267 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T07:10:49.283 INFO:teuthology.orchestra.run.smithi044.stdout: 262213 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest 2024-04-22T07:10:49.283 INFO:teuthology.orchestra.run.smithi044.stdout: 410581 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 22 07:06 /home/ubuntu/cephtest/mnt.0 2024-04-22T07:10:49.283 INFO:teuthology.orchestra.run.smithi044.stdout: 410582 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 22 07:06 /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T07:10:49.283 INFO:teuthology.orchestra.run.smithi044.stdout: 410583 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 22 07:06 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T07:10:49.284 INFO:teuthology.orchestra.run.smithi044.stdout: 410585 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:06 /home/ubuntu/cephtest/mnt.0/client.0/tmp/foo 2024-04-22T07:10:49.284 INFO:teuthology.orchestra.run.smithi044.stdout: 410586 64716 -rw-r--r-- 1 ubuntu ubuntu 66266488 Apr 22 07:06 /home/ubuntu/cephtest/mnt.0/client.0/tmp/foo/linux-2.6.33.tar.bz2 2024-04-22T07:10:49.284 INFO:teuthology.orchestra.run.smithi044.stdout: 410584 64716 -rw-r--r-- 1 ubuntu ubuntu 66266488 Feb 24 2010 /home/ubuntu/cephtest/mnt.0/client.0/tmp/linux-2.6.33.tar.bz2 2024-04-22T07:10:49.284 INFO:teuthology.orchestra.run.smithi044.stdout: 262214 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive 2024-04-22T07:10:49.284 INFO:teuthology.orchestra.run.smithi044.stdout: 396268 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive/audit 2024-04-22T07:10:49.284 INFO:teuthology.orchestra.run.smithi044.stdout: 396270 128 -rw------- 1 ubuntu ubuntu 127869 Apr 22 07:10 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-04-22T07:10:49.284 INFO:teuthology.orchestra.run.smithi044.stdout: 262216 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive/syslog 2024-04-22T07:10:49.284 INFO:teuthology.orchestra.run.smithi044.stdout: 262259 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:57 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-04-22T07:10:49.284 INFO:teuthology.orchestra.run.smithi044.stdout: 262217 184 -rw-r--r-- 1 ubuntu ubuntu 187101 Apr 22 07:10 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:10:49.285 INFO:teuthology.orchestra.run.smithi044.stdout: 262218 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:57 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-04-22T07:10:49.285 INFO:teuthology.orchestra.run.smithi044.stdout: 262215 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:49.285 INFO:teuthology.orchestra.run.smithi044.stdout: 263987 29040 -rw------- 1 root root 262037504 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump/1713769536.39430.core 2024-04-22T07:10:49.285 INFO:teuthology.orchestra.run.smithi044.stdout: 263988 30900 -rw------- 1 root root 261820416 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump/1713769536.39429.core 2024-04-22T07:10:49.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T07:10:49.286 INFO:teuthology.orchestra.run.smithi044.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-22T07:10:49.287 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 310, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 260, in stop_all_quiescers thrasher.stop() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 252, in stop self.tell_quiesce_leader( "--cancel", "--all" ) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/quiescer.py", line 110, in tell_quiesce_leader leader = self.fs.get_var('qdb_leader') File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 959, in get_var return self.get_mds_map(status=status)[var] File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 955, in get_mds_map status = self.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 408, in status return FSStatus(self.mon_manager, epoch) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/cephfs/filesystem.py", line 88, in __init__ self.map = json.loads(self.get_ceph_cmd_stdout(*cmd)) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout return self.run_ceph_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd return self.mon_manager.run_cluster_cmd(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_86f7587a5a09af35f0895e1a2d08527638fae697/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi044 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-22T07:10:49.300 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-22T07:10:49.310 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-22T07:10:49.320 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi044 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' 2024-04-22T07:10:49.320 INFO:teuthology.nuke:Checking targets against current locks 2024-04-22T07:10:49.336 INFO:teuthology.orchestra.run.smithi070.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest 2024-04-22T07:10:49.336 INFO:teuthology.orchestra.run.smithi070.stdout: 262214 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive 2024-04-22T07:10:49.336 INFO:teuthology.orchestra.run.smithi070.stdout: 395989 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive/audit 2024-04-22T07:10:49.336 INFO:teuthology.orchestra.run.smithi070.stdout: 396272 112 -rw------- 1 ubuntu ubuntu 113491 Apr 22 07:10 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-04-22T07:10:49.337 INFO:teuthology.orchestra.run.smithi070.stdout: 262216 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:10 /home/ubuntu/cephtest/archive/syslog 2024-04-22T07:10:49.337 INFO:teuthology.orchestra.run.smithi070.stdout: 262259 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:57 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-04-22T07:10:49.337 INFO:teuthology.orchestra.run.smithi070.stdout: 262217 180 -rw-r--r-- 1 ubuntu ubuntu 182343 Apr 22 07:10 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T07:10:49.337 INFO:teuthology.orchestra.run.smithi070.stdout: 262218 4 -rw-rw-rw- 1 ubuntu ubuntu 29 Apr 22 06:57 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-04-22T07:10:49.337 INFO:teuthology.orchestra.run.smithi070.stdout: 262215 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump 2024-04-22T07:10:49.337 INFO:teuthology.orchestra.run.smithi070.stdout: 262284 28400 -rw------- 1 root root 261726208 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump/1713769536.39103.core 2024-04-22T07:10:49.337 INFO:teuthology.orchestra.run.smithi070.stdout: 262283 27900 -rw------- 1 root root 261341184 Apr 22 07:05 /home/ubuntu/cephtest/archive/coredump/1713769535.39104.core 2024-04-22T07:10:49.338 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-22T07:10:49.361 DEBUG:teuthology.nuke:shortname: smithi044 2024-04-22T07:10:49.361 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T07:10:49.407 DEBUG:teuthology.nuke:shortname: smithi070 2024-04-22T07:10:49.407 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T07:10:49.419 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi044.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:46.230443', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJW4DH9/OuIrQPDDxiqj26fgsnmpfYkm+x/wb5NNR8Y5p1CnQ3FLcnYlHCQjc5E30uENfANhCb/KTxaQcwDa54w='} 2024-04-22T07:10:49.465 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_05:52:53-fs-wip-lusov-quiescer-distro-default-smithi/7668157', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 06:51:46.231846', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGdaKEfSykVzVIZHdC406/XHcSmPSaAcpxjPhJfeCQCrhbd2qVl9nSXdPpBrNIGCrpVzfbG/iCeBBGWamxCjO4k='} 2024-04-22T07:10:49.513 INFO:teuthology.orchestra.console.smithi044:Power off 2024-04-22T07:10:49.513 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T07:10:49.549 INFO:teuthology.orchestra.console.smithi070:Power off 2024-04-22T07:10:49.550 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T07:10:49.559 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2024-04-22T07:10:49.559 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:49.585 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2024-04-22T07:10:49.586 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:49.607 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-22T07:10:49.620 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-04-22T07:10:53.608 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:53.621 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:53.719 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-22T07:10:53.731 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2024-04-22T07:10:57.721 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:57.768 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:10:57.832 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-04-22T07:10:57.878 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2024-04-22T07:10:57.979 INFO:teuthology.orchestra.console.smithi070:Power off completed 2024-04-22T07:11:01.837 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T07:11:01.947 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-04-22T07:11:02.048 INFO:teuthology.orchestra.console.smithi044:Power off completed 2024-04-22T07:11:02.154 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/with-quiesce 2-workunit/fs/snaps}} duration: 789.3284847736359 failure_reason: 'Command failed (workunit test fs/snaps/snap-rm-diff.sh) on smithi044 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86f7587a5a09af35f0895e1a2d08527638fae697 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=fb091e6d9896408dba3fbece7b86b757 status: fail success: false 2024-04-22T07:11:02.155 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T07:11:02.249 INFO:teuthology.run:FAIL